Skip to content

Commit

Permalink
Fix thread leak in Python 3.7 #340 (#356)
Browse files Browse the repository at this point in the history
* Fix thread leak in Python 3.7 #340

The leak is caused by the fact that in Python 3.7, the default behavior
of the `ThreadingMixin` is to use no daemon threads, but to request to
block on threads on close. Because of that, it collects references to
all created threads, creating the "leak":
https://github.com/python/cpython/blob/v3.7.0/Lib/socketserver.py#L661

* Python 3.7: `block_on_close` is `True`: https://github.com/python/cpython/blob/v3.7.0/Lib/socketserver.py#L635
* Python 3.6: `_block_on_close` is `False`: https://github.com/python/cpython/blob/v3.6.7/Lib/socketserver.py#L639
* Python 2.7: There is no `block_on_close`, thus no logic for collecting
  references: https://github.com/python/cpython/blob/v2.7.15/Lib/SocketServer.py#L582

Fix by setting `daemon_threads` to `True`, which in our case should be a
reasonable setting for all Python versions. Also, the new in Python 3.7
`ThreadingHTTPServer` stdlib class also sets it by default:
https://github.com/python/cpython/blob/v3.7.0/Lib/http/server.py#L144

Signed-off-by: Sebastian Brandt <sebastian.brandt@friday.de>
  • Loading branch information
sbrandtb authored and brian-brazil committed Dec 11, 2018
1 parent d7c70f2 commit 5aa256d
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions prometheus_client/exposition.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,12 @@ def factory(cls, registry):

class _ThreadingSimpleServer(ThreadingMixIn, HTTPServer):
"""Thread per request HTTP server."""
# Make worker threads "fire and forget". Beginning with Python 3.7 this
# prevents a memory leak because ``ThreadingMixIn`` starts to gather all
# non-daemon threads in a list in order to join on them at server close.
# Enabling daemon threads virtually makes ``_ThreadingSimpleServer`` the
# same as Python 3.7's ``ThreadingHTTPServer``.
daemon_threads = True


def start_http_server(port, addr='', registry=REGISTRY):
Expand Down

0 comments on commit 5aa256d

Please sign in to comment.