Data corruption when multiple parsers are run simultaneously on the same chain

[hkalodner]

Please provide a clear and concise description of what the problem is.

Expected Result

blocksci_parser should detect if another process is already operating on the same directory

Actual Result

There's so safeguard in place so 2 processes can corrupt the data

Reproduction Steps

Run the parser twice simultaneously

System Information

N/A

[maltemoeser]

@hkalodner this seems like it could be an issue for the v0.5 AMI. If the node updates much faster than the parser it could happen that another cronjob is started while the previous parser process is still running, potentially leading to data corruption

[maltemoeser]

Looks like currently on the AMI a second parser process will stop due to a lock put in place by the database.

terminate called after throwing an instance of 'std::runtime_error'
  what():  Could not open hash index with error: While lock file: /home/ubuntu/bitcoin/hashIndex/LOCK: Resource temporarily unavailable

[jiagengliu]

It might sound silly, but I accidentally stopped the parser process and now the parser complains that a PID file already exists and aborts. Is there a way to clean up and rerun? Thank you.

[maltemoeser]

@jiagengliu If the parser was interrupted, I'd strongly recommend to reparse the whole chain since there may be data corruption (#33). You could simply delete the PID file in the data directory to continue parsing, but I don't recommend it.