Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError: 731 #117

Closed
SeyZ opened this issue Jul 9, 2012 · 2 comments
Closed

KeyError: 731 #117

SeyZ opened this issue Jul 9, 2012 · 2 comments
Labels

Comments

@SeyZ
Copy link

SeyZ commented Jul 9, 2012

First of all, thanks for this awesome project. It's really useful.

When I use watchdog intensively, I have sometime this kind of exception:

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/home/seyz/workspace/virtenvs/foo/lib/python2.7/site-packages/watchdog/observers/api.py", line 191, in run
    self.queue_events(self.timeout)
  File "/home/seyz/workspace/virtenvs/foo/lib/python2.7/site-packages/watchdog/observers/inotify.py", line 748, in queue_events
    inotify_events = self._inotify.read_events()
  File "/home/seyz/workspace/virtenvs/foo/lib/python2.7/site-packages/watchdog/observers/inotify.py", line 538, in read_events
    wd_path = self._path_for_wd[wd]
KeyError: 731

Edit issue:

Here's exactly the same stacktrace (with the error at the same line) but the KeyError is now 730. Could you help me ?

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/home/seyz/workspace/virtenvs/foo/lib/python2.7/site-packages/watchdog/observers/api.py", line 191, in run
    self.queue_events(self.timeout)
  File "/home/seyz/workspace/virtenvs/foo/lib/python2.7/site-packages/watchdog/observers/inotify.py", line 748, in queue_events
    inotify_events = self._inotify.read_events()
  File "/home/seyz/workspace/virtenvs/foo/lib/python2.7/site-packages/watchdog/observers/inotify.py", line 538, in read_events
    wd_path = self._path_for_wd[wd]
KeyError: 730
@danilobellini
Copy link
Collaborator

That's not fixed yet, and it isn't so random as it perhaps seems. I'm having this issue "consistently" here. I mean, all the time (on every run), unless I add an artificial 50ms sleep to the event handler:

Exception in thread Thread-5:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/site-packages/watchdog/observers/inotify_buffer.py", line 59, in run
    inotify_events = self._inotify.read_events()
  File "/usr/lib/python2.7/site-packages/watchdog/observers/inotify_c.py", line 307, in read_events
    wd_path = self._path_for_wd[wd]
KeyError: 966

@danilobellini
Copy link
Collaborator

I was checking here the cause, as it seems it's due to a path that appears twice in the self._path_for_wd. In the execution order, Inotify.read_events calls _remove_watch_bookkeeping to remove the wd=964, but it removes the wd=966 that seem to have the same source path:

for loop: wd=964, name=
caller (when inotify_event.is_ignored): wd=964, src_path=./.tox/log
_remove_watch_bookkeeping: wd=966, del _path_for_wd=./.tox/log
for loop: wd=966, name=tox-0.log
[... crashes ...]

Should I open a new issue?

danilobellini added a commit to danilobellini/watchdog that referenced this issue Sep 24, 2016
- This is related to gorakhargosh#117 and gorakhargosh#233
- It's about a single path that is has 2 wd values, i.e., a
  subdirectory path that was created again before the "bookkeeping
  removal" event (IN_IGNORE) of a deletion was processed
- The test is using a stochastical approach to reach the bug, in the
  sense that being "fast" isn't enough to enforce a particular event
  order; however, a "lucky" event synchronization seems extremely rare
  and the package should pass on every case
- Even with an unknown order for the events, the test is counting and
  enforces:
  - Watchdog emits the 120 events that happened
  - Specific number of events for each type
  - Order constraing as required for minimal consistency (e.g. not
    deleting a directory that wasn't created)
danilobellini added a commit to danilobellini/watchdog that referenced this issue Sep 24, 2016
- The call on inotify_event.is_ignored was removing the wrong wd, due
  to another wd with the same path (self._wd_for_path[src_path] was
  replaced)
- Fix gorakhargosh#117 and gorakhargosh#233
- Still not enough to fix every bug regarding fast/unordered events,
  but doesn't crash anymore
DonyorM pushed a commit that referenced this issue Oct 26, 2016
Inotify event order/duplication, fix #117 and #233
CCP-Aporia pushed a commit to CCP-Aporia/watchdog that referenced this issue Aug 13, 2020
CCP-Aporia pushed a commit to CCP-Aporia/watchdog that referenced this issue Aug 13, 2020
- This is related to gorakhargosh#117 and gorakhargosh#233
- It's about a single path that is has 2 wd values, i.e., a
  subdirectory path that was created again before the "bookkeeping
  removal" event (IN_IGNORE) of a deletion was processed
- The test is using a stochastical approach to reach the bug, in the
  sense that being "fast" isn't enough to enforce a particular event
  order; however, a "lucky" event synchronization seems extremely rare
  and the package should pass on every case
- Even with an unknown order for the events, the test is counting and
  enforces:
  - Watchdog emits the 120 events that happened
  - Specific number of events for each type
  - Order constraing as required for minimal consistency (e.g. not
    deleting a directory that wasn't created)
CCP-Aporia pushed a commit to CCP-Aporia/watchdog that referenced this issue Aug 13, 2020
- The call on inotify_event.is_ignored was removing the wrong wd, due
  to another wd with the same path (self._wd_for_path[src_path] was
  replaced)
- Fix gorakhargosh#117 and gorakhargosh#233
- Still not enough to fix every bug regarding fast/unordered events,
  but doesn't crash anymore
CCP-Aporia pushed a commit to CCP-Aporia/watchdog that referenced this issue Aug 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants