fix: Add try/except for jobs_to_keep deletions, closes #159

scrapy · Jul 15, 2024 · d0d3769 · d0d3769
1 parent 8ba341b
commit d0d3769
Show file tree

Hide file tree

Showing 2 changed files with 5 additions and 1 deletion.
diff --git a/docs/news.rst b/docs/news.rst
@@ -25,6 +25,7 @@ Fixed
 ~~~~~
 
 - The :ref:`cancel.json` webservice now works on Windows.
+- The :ref:`jobs_to_keep` setting no longer causes an error if a file to delete can't be deleted (for example, if the file is open on Windows).
 - When managing multiple projects, the next pending job for all but one project was unreported by the :ref:`daemonstatus.json` and :ref:`listjobs.json` webservices, and was not cancellable by the :ref:`cancel.json` webservice.
 
 1.4.3 (2023-09-25)

diff --git a/scrapyd/environ.py b/scrapyd/environ.py
@@ -70,5 +70,8 @@ def _get_file(self, message, dir, ext):
             key=os.path.getmtime,
         )[:-self.jobs_to_keep]
         for x in to_delete:
-            os.remove(x)
+            try:
+                os.remove(x)
+            except OSError:
+                pass
         return os.path.join(absdir, "%s.%s" % (message['_job'], ext))