Merge ff2c315 into staging

README.md

@@ -12,13 +12,13 @@ draft.**
 `codn` encrypts data entries. Entries can be added, updated, and removed.
 Entries are files or strings.
 
-Each entry is independent and opens with a unique **codename**.
-
-Knowing one codename reveals nothing about other entries. Neither the user nor
-the utility has that information. Everything in the vault either
-cryptographically secure or overly obfuscated. There is no table of contents and
-no master decryption keys.
+Each entry is independent and protected with a unique **codename**. The codename
+serves as a name and password at the same time.
 
+Codename allows access to one entry. It reveals nothing about other entries,
+even whether they exist. Neither the user nor `codn` has that information. The
+vault is cryptographically secure and overly obfuscated. There is no table of
+contents and no master decryption keys.
 
 # Install
 
@@ -90,25 +90,46 @@ My lover's jokes are not that funny
 
 # Under the hood
 
-## Obfuscation
+- Entries are encrypted really well
+- Number of entries cannot be determined
+
+## Entries obfuscation
 
 `codn` stores encrypted entries inside blobs. The number and size of blobs is no
 secret. Their contents are secret.
 
-- Which blobs refer to the same codename is unknown and cryptographically hidden
+- The number of blobs is random. Many blobs are fake. They are indistinguishable
+  from real data, but do not contain anything meaningful
 
 - The blob sizes are random. They are unrelated to the size of the entries.
   Large entries are broken into parts, and small ones are padded
 
-- Many blobs are fake. They are indistinguishable from real data, but do not
-  contain anything meaningful
+- Which blobs refer to the same codename is unknown. We can only determine blobs
+  associated with a particular codename if the user provided this codename
 
 - Random actions are taken every time the vault is updated: some fake blobs are
   added, and some are removed
 
+Thus, **number and size of entries cannot be determined** by the size of the
+vault file or number of blobs.
+
+The payload is smaller than the vault size. Only this is known for certain.
+
+## File obfuscation
+
 The file itself, at first glance, does not have format-identifying information,
 and does not have any evident structure.
 
+For example, in a regular binary file, the 32-bit number 42 looks
+like `00 00 00 2A`. In an obfuscated `codn` file, the same number would be
+something like `F8 70 4A 52`. Thus, literally all bytes appear to be encrypted.
+Even the first four bytes identifying the `codn` format are different each time.
+
+It is worth clarifying that this obfuscation is mainly decorative. You can 
+check if a file is in the `codn` format with the `codn` utility, if you aware 
+of it.
+
+
 ## Encryption
 
 1) **URandom** creates 256-bit **salt** when we initialize the vault file. The

codn/b_storage_file/_20_blobs_list_io.py

@@ -8,10 +8,20 @@
 import zlib
 from typing import BinaryIO, Tuple, Optional, List, Iterable
 
+from codn._common import read_or_fail, InsufficientData
 from codn.b_cryptoblobs._10_byte_funcs import uint32_to_bytes, \
     bytes_to_uint32, uint16_to_bytes, bytes_to_uint16
 from codn.b_storage_file._10_fragment_io import FragmentIO
-from codn._common import read_or_fail, InsufficientData
+
+
+def _obfuscate_size(size16: int, crc32: int) -> int:
+    if not 0 <= size16 <= 0xFFFF:
+        raise ValueError(f"size: {size16}")
+    if not 0 <= crc32 <= 0xFFFFFFFF:
+        raise ValueError(f"crc32: {crc32}")
+
+    mix_mask = (crc32 >> 16) ^ crc32
+    return (size16 ^ mix_mask) & 0xFFFF
 
 
 class BlobsSequentialWriter:
@@ -135,7 +145,7 @@ def io(self, idx: int) -> FragmentIO:
         return FragmentIO(frio.underlying, frio.start, frio.length)
 
         # frio.seek(0, io.SEEK_SET)
-        #return frio
+        # return frio
 
     # def get_bytes(self, idx: ):
 

tests/test_blobs_list_file.py

@@ -1,15 +1,32 @@
 import io
+import random
 import unittest
 import zlib
 from io import BytesIO
 
 from codn.b_storage_file._20_blobs_list_io import BlobsSequentialWriter, \
     BlobsSequentialReader, BlobChecksumMismatch, \
-    BlobsIndexedReader
+    BlobsIndexedReader, _obfuscate_size
 
 
 class TestBlobsListFile(unittest.TestCase):
 
+    def test_obfuscate_size_const(self):
+
+        x = 0x000A
+        obfuscated = 0xF4A3
+        crc = 0x1B83EF2A
+
+        self.assertEqual(_obfuscate_size(_obfuscate_size(x, crc), crc), x)
+        self.assertEqual(_obfuscate_size(x, crc), obfuscated)
+
+    def test_obfuscate_size_list(self):
+
+        for x in [0x0000, 0x1234, 0xFFFF]:
+            for _ in range(3):
+                crc = random.randint(0, 0xFFFFFFFF)
+                self.assertEqual(_obfuscate_size(_obfuscate_size(x, crc), crc),
+                                 x)
 
     def test_write_read_bytes(self):
         with BytesIO() as large_io:
@@ -65,8 +82,6 @@ def test_indexed_reader(self):
             large_io.seek(blobs_start_idx, io.SEEK_SET)
             bir = BlobsIndexedReader(large_io)
 
-            #self.fail()
-
             with self.subTest("Read content"):
                 self.assertEqual(bir.io(1).read(), b'hello')
                 self.assertEqual(bir.io(3).read(), b'777')