Multiprocess mode: using direct assignment for writing into mmap

[Heldroe]

@brian-brazil

Using struct.pack_into can create atomicity problems when writing on CPython, setting the bytes to 0 before writing the value.

When having a lot of writes, some reads would return 0 instead of the real value. This line in CPython is suspected: https://github.com/python/cpython/blob/2.7/Modules/_struct.c#L1563

You can reproduce the issue by having two scripts running in two different processes but using the same file, one writing an always incrementing counter and the other reading it. Sometimes, the reader script will read 0 instead of the real value. We encountered this in production by running a Django web service behind gunicorn.

Example writer script:

#!/usr/bin/python

import time
from prometheus_client.core import _MmapedDict


f = _MmapedDict('mf.db')
start = time.time()
iterations = 0

while True:
    iterations += 1
    f.write_value('v', float(iterations))
    if iterations % 500000 == 0:
        elapsed = time.time() - start
        rate = int(iterations / elapsed)
        print("{elapsed}s\t{iterations} it\t{rate} it/s".format(elapsed=elapsed, iterations=iterations, rate=rate))

Example reader script:

#!/usr/bin/python

import time
from prometheus_client.core import _MmapedDict


f = _MmapedDict('mf.db', read_mode=True)
start = time.time()
iterations = 0
errors = 0

while True:
    values = list(f._read_all_values())
    iterations += 1
    if values[0][1] == 0:
        errors += 1
    if iterations % 100000 == 0:
        elapsed = time.time() - start
        rate = int(iterations / elapsed)
        print("{elapsed}s\t{iterations} it\t{rate} it/s (v={value}, e={errors})".format(
            elapsed=elapsed,
            iterations=iterations,
            rate=rate,
            value=values[0][1],
            errors=errors,
        ))

If you execute both of these at the same time, you will see the number of 0 reads increasing. This PR fixes the issue.

[brian-brazil]

Thanks, can you add a comment so this doesn't get accidentally reverted in future?

[brian-brazil]

Is there the same issue with this?

[Heldroe]

Didn't test in details but I would assume so. I'll apply the same fix there.

[brian-brazil]

Thanks!