You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It appears (but is not proven), than in our environment when the s3 storage server is under extreme load, that one-block files go missing. The working theory is that having multiple deep-mind processes writing to the same file to the same location causes some race condition and the file just disappears.
Possible workaround: have each of the mindreaders write to its own location and then have the merger read from all those locations.
The text was updated successfully, but these errors were encountered:
matthewdarwin
changed the title
Feature: allow merger to read from multiple locations
Feature: allow merger to read from multiple one-block locations
Sep 2, 2020
It appears (but is not proven), than in our environment when the s3 storage server is under extreme load, that one-block files go missing. The working theory is that having multiple deep-mind processes writing to the same file to the same location causes some race condition and the file just disappears.
Possible workaround: have each of the mindreaders write to its own location and then have the merger read from all those locations.
The text was updated successfully, but these errors were encountered: