<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Periodic Executors — PyMongo 3.7.2 documentation</title>
<link rel="stylesheet" href="../_static/pydoctheme.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/language_data.js"></script>
<script type="text/javascript" src="../_static/sidebar.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="prev" title="Developer Guide" href="index.html" />
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="index.html" title="Developer Guide"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PyMongo 3.7.2 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Developer Guide</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="periodic-executors">
<h1>Periodic Executors<a class="headerlink" href="#periodic-executors" title="Permalink to this headline">¶</a></h1>
<p>PyMongo implements a <code class="xref py py-class docutils literal notranslate"><span class="pre">PeriodicExecutor</span></code> for two
purposes: as the background thread for <code class="xref py py-class docutils literal notranslate"><span class="pre">Monitor</span></code>, and to
regularly check if there are <cite>OP_KILL_CURSORS</cite> messages that must be sent to the server.</p>
<div class="section" id="killing-cursors">
<h2>Killing Cursors<a class="headerlink" href="#killing-cursors" title="Permalink to this headline">¶</a></h2>
<p>An incompletely iterated <a class="reference internal" href="../api/pymongo/cursor.html#pymongo.cursor.Cursor" title="pymongo.cursor.Cursor"><code class="xref py py-class docutils literal notranslate"><span class="pre">Cursor</span></code></a> on the client represents an
open cursor object on the server. In code like this, we lose a reference to
the cursor before finishing iteration:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">for</span> <span class="n">doc</span> <span class="ow">in</span> <span class="n">collection</span><span class="o">.</span><span class="n">find</span><span class="p">():</span>
<span class="k">raise</span> <span class="ne">Exception</span><span class="p">()</span>
</pre></div>
</div>
<p>We try to send an <cite>OP_KILL_CURSORS</cite> to the server to tell it to clean up the
server-side cursor. But we must not take any locks directly from the cursor’s
destructor (see <a class="reference external" href="https://jira.mongodb.org/browse/PYTHON-799">PYTHON-799</a>), so we cannot safely use the PyMongo data
structures required to send a message. The solution is to add the cursor’s id
to an array on the <a class="reference internal" href="../api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient" title="pymongo.mongo_client.MongoClient"><code class="xref py py-class docutils literal notranslate"><span class="pre">MongoClient</span></code></a> without taking any locks.</p>
<p>Each client has a <code class="xref py py-class docutils literal notranslate"><span class="pre">PeriodicExecutor</span></code> devoted to
checking the array for cursor ids. Any it sees are the result of cursors that
were freed while the server-side cursor was still open. The executor can safely
take the locks it needs in order to send the <cite>OP_KILL_CURSORS</cite> message.</p>
</div>
<div class="section" id="stopping-executors">
<h2>Stopping Executors<a class="headerlink" href="#stopping-executors" title="Permalink to this headline">¶</a></h2>
<p>Just as <a class="reference internal" href="../api/pymongo/cursor.html#pymongo.cursor.Cursor" title="pymongo.cursor.Cursor"><code class="xref py py-class docutils literal notranslate"><span class="pre">Cursor</span></code></a> must not take any locks from its destructor,
neither can <a class="reference internal" href="../api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient" title="pymongo.mongo_client.MongoClient"><code class="xref py py-class docutils literal notranslate"><span class="pre">MongoClient</span></code></a> and <code class="xref py py-class docutils literal notranslate"><span class="pre">Topology</span></code>.
Thus, although the client calls <code class="xref py py-meth docutils literal notranslate"><span class="pre">close()</span></code> on its kill-cursors thread, and
the topology calls <code class="xref py py-meth docutils literal notranslate"><span class="pre">close()</span></code> on all its monitor threads, the <code class="xref py py-meth docutils literal notranslate"><span class="pre">close()</span></code>
method cannot actually call <code class="xref py py-meth docutils literal notranslate"><span class="pre">wake()</span></code> on the executor, since <code class="xref py py-meth docutils literal notranslate"><span class="pre">wake()</span></code>
takes a lock.</p>
<p>Instead, executors wake periodically to check if <code class="docutils literal notranslate"><span class="pre">self.close</span></code> is set,
and if so they exit.</p>
<p>A thread can log spurious errors if it wakes late in the Python interpreter’s
shutdown sequence, so we try to join threads before then. Each periodic
executor (either a monitor or a kill-cursors thread) adds a weakref to itself
to a set called <code class="docutils literal notranslate"><span class="pre">_EXECUTORS</span></code>, in the <code class="docutils literal notranslate"><span class="pre">periodic_executor</span></code> module.</p>
<p>An <a class="reference external" href="https://docs.python.org/2/library/atexit.html">exit handler</a> runs on shutdown and tells all executors to stop, then
tries (with a short timeout) to join all executor threads.</p>
</div>
<div class="section" id="monitoring">
<h2>Monitoring<a class="headerlink" href="#monitoring" title="Permalink to this headline">¶</a></h2>
<p>For each server in the topology, <code class="xref py py-class docutils literal notranslate"><span class="pre">Topology</span></code> uses a periodic
executor to launch a monitor thread. This thread must not prevent the topology
from being freed, so it weakrefs the topology. Furthermore, it uses a weakref
callback to terminate itself soon after the topology is freed.</p>
<p>Solid lines represent strong references, dashed lines weak ones:</p>
<img alt="../_images/periodic-executor-refs.png" src="../_images/periodic-executor-refs.png" />
<p>See <a class="reference internal" href="#stopping-executors">Stopping Executors</a> above for an explanation of the <code class="docutils literal notranslate"><span class="pre">_EXECUTORS</span></code> set.</p>
<p>It is a requirement of the <a class="reference external" href="https://github.com/mongodb/specifications/blob/master/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst#requesting-an-immediate-check">Server Discovery And Monitoring Spec</a> that a
sleeping monitor can be awakened early. Aside from infrequent wakeups to do
their appointed chores, and occasional interruptions, periodic executors also
wake periodically to check if they should terminate.</p>
<p>Our first implementation of this idea was the obvious one: use the Python
standard library’s threading.Condition.wait with a timeout. Another thread
wakes the executor early by signaling the condition variable.</p>
<p>A topology cannot signal the condition variable to tell the executor to
terminate, because it would risk a deadlock in the garbage collector: no
destructor or weakref callback can take a lock to signal the condition variable
(see <a class="reference external" href="https://jira.mongodb.org/browse/PYTHON-863">PYTHON-863</a>); thus the only way for a dying object to terminate a
periodic executor is to set its “stopped” flag and let the executor see the
flag next time it wakes.</p>
<p>We erred on the side of prompt cleanup, and set the check interval at 100ms. We
assumed that checking a flag and going back to sleep 10 times a second was
cheap on modern machines.</p>
<p>Starting in Python 3.2, the builtin C implementation of lock.acquire takes a
timeout parameter, so Python 3.2+ Condition variables sleep simply by calling
lock.acquire; they are implemented as efficiently as expected.</p>
<p>But in Python 2, lock.acquire has no timeout. To wait with a timeout, a Python
2 condition variable sleeps a millisecond, tries to acquire the lock, sleeps
twice as long, and tries again. This exponential backoff reaches a maximum
sleep time of 50ms.</p>
<p>If PyMongo calls the condition variable’s “wait” method with a short timeout,
the exponential backoff is restarted frequently. Overall, the condition variable
is not waking a few times a second, but hundreds of times. (See <a class="reference external" href="https://jira.mongodb.org/browse/PYTHON-983">PYTHON-983</a>.)</p>
<p>Thus the current design of periodic executors is surprisingly simple: they
do a simple <cite>time.sleep</cite> for a half-second, check if it is time to wake or
terminate, and sleep again.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Periodic Executors</a><ul>
<li><a class="reference internal" href="#killing-cursors">Killing Cursors</a></li>
<li><a class="reference internal" href="#stopping-executors">Stopping Executors</a></li>
<li><a class="reference internal" href="#monitoring">Monitoring</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="index.html"
title="previous chapter">Developer Guide</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/developer/periodic_executor.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="index.html" title="Developer Guide"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PyMongo 3.7.2 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="index.html" >Developer Guide</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright MongoDB, Inc. 2008-present. MongoDB, Mongo, and the leaf logo are registered trademarks of MongoDB, Inc.
</div>
</body>
</html>
distrib
>
Mageia
>
7
>
armv7hl
>
media
>
core-release
>
by-pkgid
>
bf9ddee8352dabf2ba1d26636a349741
>
files
>
418
