<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Bcfg2.Server.Plugins.Packages.Source — Bcfg2 1.3.0 documentation</title>
<link rel="stylesheet" href="../../../../../_static/default.css" type="text/css" />
<link rel="stylesheet" href="../../../../../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../../../../../',
VERSION: '1.3.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../../../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../../../_static/sidebar.js"></script>
<link rel="shortcut icon" href="../../../../../_static/favicon.ico"/>
<link rel="top" title="Bcfg2 1.3.0 documentation" href="../../../../../index.html" />
<link rel="up" title="Bcfg2.Server.Plugins.Packages" href="../Packages.html" />
<link rel="stylesheet" href="../../../../../_static/bcfg2.css" type=""/>
</head>
<body>
<div style="text-align: left; padding: 10px 10px 15px 15px">
<a href="../../../../../index.html"><img src="../../../../../_static/bcfg2_logo.png" border="0" alt="sampledoc"/></a>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../../../../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../../../../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li><a href="../../../../../index.html">home</a> | </li>
<!--<li><a href="../../../../../search.html">search</a> | </li>-->
<li><a href="../../../../../help/index.html">help</a> | </li>
<li><a href="../../../../../contents.html">documentation </a> »</li>
<li><a href="../../../../index.html" >Module code</a> »</li>
<li><a href="../../Plugins.html" >Bcfg2.Server.Plugins</a> »</li>
<li><a href="../Packages.html" accesskey="U">Bcfg2.Server.Plugins.Packages</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<h1>Source code for Bcfg2.Server.Plugins.Packages.Source</h1><div class="highlight"><pre>
<span class="sd">""" ``Source`` objects represent a single <Source> tag in</span>
<span class="sd">``sources.xml``. Note that a single Source tag can itself describe</span>
<span class="sd">multiple repositories (if it uses the "url" attribute instead of</span>
<span class="sd">"rawurl"), and so can the ``Source`` object. This can be the source</span>
<span class="sd">(har har) of some confusion. See</span>
<span class="sd">:func:`Bcfg2.Server.Plugins.Packages.Collection.Collection.sourcelist`</span>
<span class="sd">for the proper way to get all repos from a ``Source`` object.</span>
<span class="sd">Source objects are aggregated into</span>
<span class="sd">:class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`</span>
<span class="sd">objects, which are actually called by</span>
<span class="sd">:class:`Bcfg2.Server.Plugins.Packages.Packages`. This way a more</span>
<span class="sd">advanced subclass can query repositories in aggregate rather than</span>
<span class="sd">individually, which may give faster or more accurate results.</span>
<span class="sd">The base ``Source`` object must be subclassed to handle each</span>
<span class="sd">repository type. How you subclass ``Source`` will depend on how you</span>
<span class="sd">subclassed</span>
<span class="sd">:class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`; see</span>
<span class="sd">:mod:`Bcfg2.Server.Plugins.Packages.Collection` for more details on</span>
<span class="sd">different methods for doing that.</span>
<span class="sd">If you are using the stock (or a near-stock)</span>
<span class="sd">:class:`Bcfg2.Server.Plugins.Packages.Collection.Collection` object,</span>
<span class="sd">then you will need to implement the following methods and attributes</span>
<span class="sd">in your ``Source`` subclass:</span>
<span class="sd">* :func:`Source.urls`</span>
<span class="sd">* :func:`Source.read_files`</span>
<span class="sd">* :attr:`Source.basegroups`</span>
<span class="sd">Additionally, you may want to consider overriding the following</span>
<span class="sd">methods and attributes:</span>
<span class="sd">* :func:`Source.is_virtual_package`</span>
<span class="sd">* :func:`Source.get_group`</span>
<span class="sd">* :attr:`Source.unknown_filter`</span>
<span class="sd">* :attr:`Source.load_state`</span>
<span class="sd">* :attr:`Source.save_state`</span>
<span class="sd">For an example of this kind of ``Source`` object, see</span>
<span class="sd">:mod:`Bcfg2.Server.Plugins.Packages.Apt`.</span>
<span class="sd">If you are overriding the ``Collection`` object in more depth, then</span>
<span class="sd">you have more leeway in what you might want to override or implement</span>
<span class="sd">in your ``Source`` subclass. For an example of this kind of</span>
<span class="sd">``Source`` object, see :mod:`Bcfg2.Server.Plugins.Packages.Yum`.</span>
<span class="sd">"""</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">Bcfg2.Server.Plugin</span>
<span class="kn">from</span> <span class="nn">Bcfg2.Compat</span> <span class="kn">import</span> <span class="n">HTTPError</span><span class="p">,</span> <span class="n">HTTPBasicAuthHandler</span><span class="p">,</span> \
<span class="n">HTTPPasswordMgrWithDefaultRealm</span><span class="p">,</span> <span class="n">install_opener</span><span class="p">,</span> <span class="n">build_opener</span><span class="p">,</span> \
<span class="n">urlopen</span><span class="p">,</span> <span class="n">cPickle</span><span class="p">,</span> <span class="n">md5</span>
<div class="viewcode-block" id="fetch_url"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.fetch_url">[docs]</a><span class="k">def</span> <span class="nf">fetch_url</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
<span class="sd">""" Return the content of the given URL.</span>
<span class="sd"> :param url: The URL to fetch content from.</span>
<span class="sd"> :type url: string</span>
<span class="sd"> :raises: ValueError - Malformed URL</span>
<span class="sd"> :raises: URLError - Failure fetching URL</span>
<span class="sd"> :returns: string - the content of the page at the given URL """</span>
<span class="k">if</span> <span class="s">'@'</span> <span class="ow">in</span> <span class="n">url</span><span class="p">:</span>
<span class="n">mobj</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s">'(\w+://)([^:]+):([^@]+)@(.*)$'</span><span class="p">,</span> <span class="n">url</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">mobj</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">"Invalid URL"</span><span class="p">)</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">mobj</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="n">passwd</span> <span class="o">=</span> <span class="n">mobj</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">mobj</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="o">+</span> <span class="n">mobj</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span>
<span class="n">auth</span> <span class="o">=</span> <span class="n">HTTPBasicAuthHandler</span><span class="p">(</span><span class="n">HTTPPasswordMgrWithDefaultRealm</span><span class="p">())</span>
<span class="n">auth</span><span class="o">.</span><span class="n">add_password</span><span class="p">(</span><span class="bp">None</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="n">user</span><span class="p">,</span> <span class="n">passwd</span><span class="p">)</span>
<span class="n">install_opener</span><span class="p">(</span><span class="n">build_opener</span><span class="p">(</span><span class="n">auth</span><span class="p">))</span>
<span class="k">return</span> <span class="n">urlopen</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</div>
<div class="viewcode-block" id="SourceInitError"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.SourceInitError">[docs]</a><span class="k">class</span> <span class="nc">SourceInitError</span><span class="p">(</span><span class="ne">Exception</span><span class="p">):</span>
<span class="sd">""" Raised when a :class:`Source` object fails instantiation. """</span>
<span class="k">pass</span>
<span class="c">#: A regular expression used to determine the base name of a repo from</span>
<span class="c">#: its URL. This is used when generating repo configs and by</span>
<span class="c">#: :func:`Source.get_repo_name`. It handles `Pulp</span>
<span class="c">#: <http://www.pulpproject.org/>`_ and `mrepo</span>
<span class="c">#: <http://dag.wieers.com/home-made/mrepo/>`_ repositories specially,</span>
<span class="c">#: and otherwise grabs the last component of the URL (as delimited by</span>
<span class="c">#: slashes).</span></div>
<span class="n">REPO_RE</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="s">r'(?:pulp/repos/|/RPMS\.|/)([^/]+)/?$'</span><span class="p">)</span>
<div class="viewcode-block" id="Source"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source">[docs]</a><span class="k">class</span> <span class="nc">Source</span><span class="p">(</span><span class="n">Bcfg2</span><span class="o">.</span><span class="n">Server</span><span class="o">.</span><span class="n">Plugin</span><span class="o">.</span><span class="n">Debuggable</span><span class="p">):</span> <span class="c"># pylint: disable=R0902</span>
<span class="sd">""" ``Source`` objects represent a single <Source> tag in</span>
<span class="sd"> ``sources.xml``. Note that a single Source tag can itself</span>
<span class="sd"> describe multiple repositories (if it uses the "url" attribute</span>
<span class="sd"> instead of "rawurl"), and so can the ``Source`` object.</span>
<span class="sd"> Note that a number of the attributes of this object may be more or</span>
<span class="sd"> less specific to one backend (e.g., :attr:`essentialpkgs`,</span>
<span class="sd"> :attr:`recommended`, :attr:`gpgkeys`, but they are included in the</span>
<span class="sd"> superclass to make the parsing of sources from XML more</span>
<span class="sd"> consistent, and to make it trivial for other backends to support</span>
<span class="sd"> those features.</span>
<span class="sd"> """</span>
<span class="c">#: The list of</span>
<span class="c">#: :ref:`server-plugins-generators-packages-magic-groups` that</span>
<span class="c">#: make sources of this type available to clients.</span>
<span class="n">basegroups</span> <span class="o">=</span> <span class="p">[]</span>
<span class="c">#: The Package type handled by this Source class. The ``type``</span>
<span class="c">#: attribute of Package entries will be set to the value ``ptype``</span>
<span class="c">#: when they are handled by :mod:`Bcfg2.Server.Plugins.Packages`.</span>
<span class="n">ptype</span> <span class="o">=</span> <span class="bp">None</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">basepath</span><span class="p">,</span> <span class="n">xsource</span><span class="p">,</span> <span class="n">setup</span><span class="p">):</span> <span class="c"># pylint: disable=R0912</span>
<span class="sd">"""</span>
<span class="sd"> :param basepath: The base filesystem path under which cache</span>
<span class="sd"> data for this source should be stored</span>
<span class="sd"> :type basepath: string</span>
<span class="sd"> :param xsource: The XML tag that describes this source</span>
<span class="sd"> :type source: lxml.etree._Element</span>
<span class="sd"> :param setup: A Bcfg2 options dict</span>
<span class="sd"> :type setup: dict</span>
<span class="sd"> :raises: :class:`Bcfg2.Server.Plugins.Packages.Source.SourceInitError`</span>
<span class="sd"> """</span>
<span class="n">Bcfg2</span><span class="o">.</span><span class="n">Server</span><span class="o">.</span><span class="n">Plugin</span><span class="o">.</span><span class="n">Debuggable</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
<span class="c">#: The base filesystem path under which cache data for this</span>
<span class="c">#: source should be stored</span>
<span class="bp">self</span><span class="o">.</span><span class="n">basepath</span> <span class="o">=</span> <span class="n">basepath</span>
<span class="c">#: The XML tag that describes this source</span>
<span class="bp">self</span><span class="o">.</span><span class="n">xsource</span> <span class="o">=</span> <span class="n">xsource</span>
<span class="c">#: A Bcfg2 options dict</span>
<span class="bp">self</span><span class="o">.</span><span class="n">setup</span> <span class="o">=</span> <span class="n">setup</span>
<span class="c">#: A set of package names that are deemed "essential" by this</span>
<span class="c">#: source</span>
<span class="bp">self</span><span class="o">.</span><span class="n">essentialpkgs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="c">#: A list of the text of all 'Component' attributes of this</span>
<span class="c">#: source from XML</span>
<span class="bp">self</span><span class="o">.</span><span class="n">components</span> <span class="o">=</span> <span class="p">[</span><span class="n">item</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'Component'</span><span class="p">)]</span>
<span class="c">#: A list of the arches supported by this source</span>
<span class="bp">self</span><span class="o">.</span><span class="n">arches</span> <span class="o">=</span> <span class="p">[</span><span class="n">item</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'Arch'</span><span class="p">)]</span>
<span class="c">#: A list of the the names of packages that are blacklisted</span>
<span class="c">#: from this source</span>
<span class="bp">self</span><span class="o">.</span><span class="n">blacklist</span> <span class="o">=</span> <span class="p">[</span><span class="n">item</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'Blacklist'</span><span class="p">)]</span>
<span class="c">#: A list of the the names of packages that are whitelisted in</span>
<span class="c">#: this source</span>
<span class="bp">self</span><span class="o">.</span><span class="n">whitelist</span> <span class="o">=</span> <span class="p">[</span><span class="n">item</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'Whitelist'</span><span class="p">)]</span>
<span class="c">#: A dict of repository options that will be included in the</span>
<span class="c">#: configuration generated on the server side (if such is</span>
<span class="c">#: applicable; most backends do not generate any sort of</span>
<span class="c">#: repository configuration on the Bcfg2 server)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">server_options</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="c">#: A dict of repository options that will be included in the</span>
<span class="c">#: configuration generated for the client (if that is</span>
<span class="c">#: supported by the backend)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">client_options</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="n">opts</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">"Options"</span><span class="p">)</span>
<span class="k">for</span> <span class="n">el</span> <span class="ow">in</span> <span class="n">opts</span><span class="p">:</span>
<span class="n">repoopts</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">([(</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">)</span>
<span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">el</span><span class="o">.</span><span class="n">attrib</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
<span class="k">if</span> <span class="n">k</span> <span class="o">!=</span> <span class="s">"clientonly"</span> <span class="ow">and</span> <span class="n">k</span> <span class="o">!=</span> <span class="s">"serveronly"</span><span class="p">])</span>
<span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"clientonly"</span><span class="p">,</span> <span class="s">"false"</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">"false"</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">server_options</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">repoopts</span><span class="p">)</span>
<span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"serveronly"</span><span class="p">,</span> <span class="s">"false"</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">"false"</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">client_options</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">repoopts</span><span class="p">)</span>
<span class="c">#: A list of URLs to GPG keys that apply to this source</span>
<span class="bp">self</span><span class="o">.</span><span class="n">gpgkeys</span> <span class="o">=</span> <span class="p">[</span><span class="n">el</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">el</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">"GPGKey"</span><span class="p">)]</span>
<span class="c">#: Whether or not to include essential packages from this source</span>
<span class="bp">self</span><span class="o">.</span><span class="n">essential</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'essential'</span><span class="p">,</span> <span class="s">'true'</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">'true'</span>
<span class="c">#: Whether or not to include recommended packages from this source</span>
<span class="bp">self</span><span class="o">.</span><span class="n">recommended</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'recommended'</span><span class="p">,</span>
<span class="s">'false'</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">'true'</span>
<span class="c">#: The "rawurl" attribute from :attr:`xsource`, if applicable.</span>
<span class="c">#: A trailing slash is automatically appended to this if there</span>
<span class="c">#: wasn't one already present.</span>
<span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'rawurl'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s">"/"</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span> <span class="o">+=</span> <span class="s">"/"</span>
<span class="c">#: The "url" attribute from :attr:`xsource`, if applicable. A</span>
<span class="c">#: trailing slash is automatically appended to this if there</span>
<span class="c">#: wasn't one already present.</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'url'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s">"/"</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">+=</span> <span class="s">"/"</span>
<span class="c">#: The "version" attribute from :attr:`xsource`</span>
<span class="bp">self</span><span class="o">.</span><span class="n">version</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'version'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span>
<span class="c">#: A list of predicates that are used to determine if this</span>
<span class="c">#: source applies to a given</span>
<span class="c">#: :class:`Bcfg2.Server.Plugins.Metadata.ClientMetadata`</span>
<span class="c">#: object.</span>
<span class="bp">self</span><span class="o">.</span><span class="n">conditions</span> <span class="o">=</span> <span class="p">[]</span>
<span class="c">#: Formerly, :ref:`server-plugins-generators-packages` only</span>
<span class="c">#: supported applying package sources to groups; that is, they</span>
<span class="c">#: could not be assigned by more complicated logic like</span>
<span class="c">#: per-client repositories and group or client negation. This</span>
<span class="c">#: attribute attempts to provide for some limited backwards</span>
<span class="c">#: compat with older code that relies on this.</span>
<span class="bp">self</span><span class="o">.</span><span class="n">groups</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">el</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">iterancestors</span><span class="p">():</span>
<span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">tag</span> <span class="o">==</span> <span class="s">"Group"</span><span class="p">:</span>
<span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"negate"</span><span class="p">,</span> <span class="s">"false"</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">"true"</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="k">lambda</span> <span class="n">m</span><span class="p">,</span> <span class="n">el</span><span class="o">=</span><span class="n">el</span><span class="p">:</span>
<span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">m</span><span class="o">.</span><span class="n">groups</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="k">lambda</span> <span class="n">m</span><span class="p">,</span> <span class="n">el</span><span class="o">=</span><span class="n">el</span><span class="p">:</span>
<span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span> <span class="ow">in</span> <span class="n">m</span><span class="o">.</span><span class="n">groups</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">el</span><span class="o">.</span><span class="n">tag</span> <span class="o">==</span> <span class="s">"Client"</span><span class="p">:</span>
<span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"negate"</span><span class="p">,</span> <span class="s">"false"</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">"true"</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="k">lambda</span> <span class="n">m</span><span class="p">,</span> <span class="n">el</span><span class="o">=</span><span class="n">el</span><span class="p">:</span>
<span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span> <span class="o">!=</span> <span class="n">m</span><span class="o">.</span><span class="n">hostname</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="k">lambda</span> <span class="n">m</span><span class="p">,</span> <span class="n">el</span><span class="o">=</span><span class="n">el</span><span class="p">:</span>
<span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span> <span class="o">==</span> <span class="n">m</span><span class="o">.</span><span class="n">hostname</span><span class="p">)</span>
<span class="c">#: A set of all package names in this source. This will not</span>
<span class="c">#: necessarily be populated, particularly by backends that</span>
<span class="c">#: reimplement large portions of</span>
<span class="c">#: :class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`</span>
<span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="c">#: A dict of ``<package name>`` -> ``<list of dependencies>``.</span>
<span class="c">#: This will not necessarily be populated, particularly by</span>
<span class="c">#: backends that reimplement large portions of</span>
<span class="c">#: :class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`</span>
<span class="bp">self</span><span class="o">.</span><span class="n">deps</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="c">#: A dict of ``<package name>`` -> ``<list of provided</span>
<span class="c">#: symbols>``. This will not necessarily be populated,</span>
<span class="c">#: particularly by backends that reimplement large portions of</span>
<span class="c">#: :class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`</span>
<span class="bp">self</span><span class="o">.</span><span class="n">provides</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="c">#: The file (or directory) used for this source's cache data</span>
<span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">basepath</span><span class="p">,</span>
<span class="s">"cache-</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">cachekey</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">:</span>
<span class="n">baseurl</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">+</span> <span class="s">"</span><span class="si">%(version)s</span><span class="s">/</span><span class="si">%(component)s</span><span class="s">/</span><span class="si">%(arch)s</span><span class="s">/"</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">baseurl</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span>
<span class="c">#: A list of dicts, each of which describes the URL to one</span>
<span class="c">#: repository contained in this source. Each dict contains</span>
<span class="c">#: the following keys:</span>
<span class="c">#:</span>
<span class="c">#: * ``version``: The version of the repo (``None`` for</span>
<span class="c">#: ``rawurl`` repos)</span>
<span class="c">#: * ``component``: The component use to form this URL</span>
<span class="c">#: (``None`` for ``rawurl`` repos)</span>
<span class="c">#: * ``arch``: The architecture of this repo</span>
<span class="c">#: * ``baseurl``: Either the ``rawurl`` attribute, or the</span>
<span class="c">#: format string built from the ``url`` attribute</span>
<span class="c">#: * ``url``: The actual URL to the repository</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url_map</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span><span class="p">:</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">:</span>
<span class="n">usettings</span> <span class="o">=</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="n">version</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">version</span><span class="p">,</span> <span class="n">component</span><span class="o">=</span><span class="n">comp</span><span class="p">,</span>
<span class="n">arch</span><span class="o">=</span><span class="n">arch</span><span class="p">)</span>
<span class="k">for</span> <span class="n">comp</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">components</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span> <span class="c"># rawurl given</span>
<span class="n">usettings</span> <span class="o">=</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="n">version</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">version</span><span class="p">,</span> <span class="n">component</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span>
<span class="n">arch</span><span class="o">=</span><span class="n">arch</span><span class="p">)]</span>
<span class="k">for</span> <span class="n">setting</span> <span class="ow">in</span> <span class="n">usettings</span><span class="p">:</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">:</span>
<span class="n">setting</span><span class="p">[</span><span class="s">'baseurl'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">setting</span><span class="p">[</span><span class="s">'baseurl'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span>
<span class="n">setting</span><span class="p">[</span><span class="s">'url'</span><span class="p">]</span> <span class="o">=</span> <span class="n">baseurl</span> <span class="o">%</span> <span class="n">setting</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url_map</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">usettings</span><span class="p">)</span>
<span class="nd">@property</span>
<div class="viewcode-block" id="Source.cachekey"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.cachekey">[docs]</a> <span class="k">def</span> <span class="nf">cachekey</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">""" A unique key for this source that will be used to generate</span>
<span class="sd"> :attr:`cachefile` and other cache paths """</span>
<span class="k">return</span> <span class="n">md5</span><span class="p">(</span><span class="n">cPickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">([</span><span class="bp">self</span><span class="o">.</span><span class="n">version</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">components</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">,</span>
<span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span><span class="p">]))</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">()</span>
</div>
<div class="viewcode-block" id="Source.get_relevant_groups"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_relevant_groups">[docs]</a> <span class="k">def</span> <span class="nf">get_relevant_groups</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span>
<span class="sd">""" Get all groups that might be relevant to determining which</span>
<span class="sd"> sources apply to this collection's client.</span>
<span class="sd"> :return: list of strings - group names</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">g</span> <span class="k">for</span> <span class="n">g</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span>
<span class="k">if</span> <span class="p">(</span><span class="n">g</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">basegroups</span> <span class="ow">or</span>
<span class="n">g</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span> <span class="ow">or</span>
<span class="n">g</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span><span class="p">)])))</span>
</div>
<div class="viewcode-block" id="Source.load_state"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.load_state">[docs]</a> <span class="k">def</span> <span class="nf">load_state</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">""" Load saved state from :attr:`cachefile`. If caching and</span>
<span class="sd"> state is handled by the package library, then this function</span>
<span class="sd"> does not need to be implemented.</span>
<span class="sd"> :raises: OSError - If the saved data cannot be read</span>
<span class="sd"> :raises: cPickle.UnpicklingError - If the saved data is corrupt """</span>
<span class="n">data</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span><span class="p">)</span>
<span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">,</span>
<span class="bp">self</span><span class="o">.</span><span class="n">essentialpkgs</span><span class="p">)</span> <span class="o">=</span> <span class="n">cPickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
</div>
<div class="viewcode-block" id="Source.save_state"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.save_state">[docs]</a> <span class="k">def</span> <span class="nf">save_state</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">""" Save state to :attr:`cachefile`. If caching and</span>
<span class="sd"> state is handled by the package library, then this function</span>
<span class="sd"> does not need to be implemented. """</span>
<span class="n">cache</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span>
<span class="n">cPickle</span><span class="o">.</span><span class="n">dump</span><span class="p">((</span><span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">,</span>
<span class="bp">self</span><span class="o">.</span><span class="n">essentialpkgs</span><span class="p">),</span> <span class="n">cache</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">cache</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</div>
<span class="nd">@Bcfg2.Server.Plugin.track_statistics</span><span class="p">()</span>
<div class="viewcode-block" id="Source.setup_data"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.setup_data">[docs]</a> <span class="k">def</span> <span class="nf">setup_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">force_update</span><span class="o">=</span><span class="bp">False</span><span class="p">):</span>
<span class="sd">""" Perform all data fetching and setup tasks. For most</span>
<span class="sd"> backends, this involves downloading all metadata from the</span>
<span class="sd"> repository, parsing it, and caching the parsed data locally.</span>
<span class="sd"> The order of operations is:</span>
<span class="sd"> #. Call :func:`load_state` to try to load data from the local</span>
<span class="sd"> cache.</span>
<span class="sd"> #. If that fails, call :func:`read_files` to read and parse</span>
<span class="sd"> the locally downloaded metadata files.</span>
<span class="sd"> #. If that fails, call :func:`update` to fetch the metadata,</span>
<span class="sd"> then :func:`read_files` to parse it.</span>
<span class="sd"> Obviously with a backend that leverages repo access libraries</span>
<span class="sd"> to avoid downloading all metadata, many of the functions</span>
<span class="sd"> called by ``setup_data`` can be no-ops (or nearly so).</span>
<span class="sd"> :param force_update: Ignore all locally cached and downloaded</span>
<span class="sd"> data and fetch the metadata anew from the</span>
<span class="sd"> upstream repository.</span>
<span class="sd"> :type force_update: bool</span>
<span class="sd"> """</span>
<span class="c"># pylint: disable=W0702</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">force_update</span><span class="p">:</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">load_state</span><span class="p">()</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Cachefile </span><span class="si">%s</span><span class="s"> load failed: </span><span class="si">%s</span><span class="s">"</span>
<span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span><span class="p">,</span> <span class="n">err</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Falling back to file read"</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">read_files</span><span class="p">()</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: File read failed: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span>
<span class="n">err</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Falling back to file download"</span><span class="p">)</span>
<span class="n">force_update</span> <span class="o">=</span> <span class="bp">True</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">force_update</span> <span class="o">=</span> <span class="bp">True</span>
<span class="k">if</span> <span class="n">force_update</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">update</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">read_files</span><span class="p">()</span>
<span class="k">except</span><span class="p">:</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Failed to load data for </span><span class="si">%s</span><span class="s">: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span>
<span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">err</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Some Packages will be missing"</span><span class="p">)</span>
<span class="c"># pylint: enable=W0702</span>
</div>
<div class="viewcode-block" id="Source.get_repo_name"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_repo_name">[docs]</a> <span class="k">def</span> <span class="nf">get_repo_name</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url_map</span><span class="p">):</span>
<span class="sd">""" Try to find a sensible name for a repository. Since</span>
<span class="sd"> ``sources.xml`` doesn't provide for repository names, we have</span>
<span class="sd"> to try to guess at the names when generating config files or</span>
<span class="sd"> doing other operations that require repository names. This</span>
<span class="sd"> function tries several approaches:</span>
<span class="sd"> #. First, if the map contains a ``component`` key, use that as</span>
<span class="sd"> the name.</span>
<span class="sd"> #. If not, then try to match the repository URL against</span>
<span class="sd"> :attr:`Bcfg2.Server.Plugins.Packages.Source.REPO_RE`. If</span>
<span class="sd"> that succeeds, use the first matched group; additionally,</span>
<span class="sd"> if the Source tag that describes this repo is contained in</span>
<span class="sd"> a ``<Group>`` tag, prepend that to the name.</span>
<span class="sd"> #. If :attr:`Bcfg2.Server.Plugins.Packages.Source.REPO_RE`</span>
<span class="sd"> does not match the repository, and the Source tag that</span>
<span class="sd"> describes this repo is contained in a ``<Group>`` tag, use</span>
<span class="sd"> the name of the group.</span>
<span class="sd"> #. Failing that, use the full URL to this repository, with the</span>
<span class="sd"> protocol and trailing slash stripped off if possible.</span>
<span class="sd"> Once that is done, all characters disallowed in yum source</span>
<span class="sd"> names are replaced by dashes. See below for the exact regex.</span>
<span class="sd"> The yum rules are used here because they are so restrictive.</span>
<span class="sd"> ``get_repo_name`` is **not** guaranteed to return a unique</span>
<span class="sd"> name. If you require a unique name, then you will need to</span>
<span class="sd"> generate all repo names and make them unique through the</span>
<span class="sd"> approach of your choice, e.g., appending numbers to non-unique</span>
<span class="sd"> repository names. See</span>
<span class="sd"> :func:`Bcfg2.Server.Plugins.Packages.Yum.Source.get_repo_name`</span>
<span class="sd"> for an example.</span>
<span class="sd"> :param url_map: A single :attr:`url_map` dict, i.e., any</span>
<span class="sd"> single element of :attr:`url_map`.</span>
<span class="sd"> :type url_map: dict</span>
<span class="sd"> :returns: string - the name of the repository.</span>
<span class="sd"> """</span>
<span class="k">if</span> <span class="n">url_map</span><span class="p">[</span><span class="s">'component'</span><span class="p">]:</span>
<span class="n">rname</span> <span class="o">=</span> <span class="n">url_map</span><span class="p">[</span><span class="s">'component'</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">match</span> <span class="o">=</span> <span class="n">REPO_RE</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">url_map</span><span class="p">[</span><span class="s">'url'</span><span class="p">])</span>
<span class="k">if</span> <span class="n">match</span><span class="p">:</span>
<span class="n">rname</span> <span class="o">=</span> <span class="n">match</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="p">:</span>
<span class="n">rname</span> <span class="o">=</span> <span class="s">"</span><span class="si">%s</span><span class="s">-</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">rname</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="p">:</span>
<span class="n">rname</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="c"># a global source with no reasonable name. Try to</span>
<span class="c"># strip off the protocol and trailing slash.</span>
<span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="s">r'^[A-z]://(.*?)/?'</span><span class="p">,</span> <span class="n">url_map</span><span class="p">[</span><span class="s">'url'</span><span class="p">])</span>
<span class="k">if</span> <span class="n">match</span><span class="p">:</span>
<span class="n">rname</span> <span class="o">=</span> <span class="n">match</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="c"># what kind of crazy url is this? I give up!</span>
<span class="c"># just use the full url and let the regex below</span>
<span class="c"># make it even uglier.</span>
<span class="n">rname</span> <span class="o">=</span> <span class="n">url_map</span><span class="p">[</span><span class="s">'url'</span><span class="p">]</span>
<span class="c"># see yum/__init__.py in the yum source, lines 441-449, for</span>
<span class="c"># the source of this regex. yum doesn't like anything but</span>
<span class="c"># string.ascii_letters, string.digits, and [-_.:]. There</span>
<span class="c"># doesn't seem to be a reason for this, because yum.</span>
<span class="k">return</span> <span class="n">re</span><span class="o">.</span><span class="n">sub</span><span class="p">(</span><span class="s">r'[^A-Za-z0-9-_.:]'</span><span class="p">,</span> <span class="s">'-'</span><span class="p">,</span> <span class="n">rname</span><span class="p">)</span>
</div>
<span class="k">def</span> <span class="nf">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">:</span>
<span class="k">return</span> <span class="s">"</span><span class="si">%s</span><span class="s"> at </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">:</span>
<span class="k">return</span> <span class="s">"</span><span class="si">%s</span><span class="s"> at </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span>
<span class="nd">@property</span>
<div class="viewcode-block" id="Source.urls"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.urls">[docs]</a> <span class="k">def</span> <span class="nf">urls</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">""" A list of URLs to the base metadata file for each</span>
<span class="sd"> repository described by this source. """</span>
<span class="k">return</span> <span class="p">[]</span>
</div>
<span class="nd">@property</span>
<div class="viewcode-block" id="Source.files"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.files">[docs]</a> <span class="k">def</span> <span class="nf">files</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">""" A list of files stored in the local cache by this backend.</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">escape_url</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">urls</span><span class="p">]</span>
</div>
<div class="viewcode-block" id="Source.get_vpkgs"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_vpkgs">[docs]</a> <span class="k">def</span> <span class="nf">get_vpkgs</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span>
<span class="sd">""" Get a list of all virtual packages provided by all sources.</span>
<span class="sd"> :returns: list of strings</span>
<span class="sd"> """</span>
<span class="n">agroups</span> <span class="o">=</span> <span class="p">[</span><span class="s">'global'</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">a</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span>
<span class="k">if</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span><span class="p">]</span>
<span class="n">vdict</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="k">for</span> <span class="n">agrp</span> <span class="ow">in</span> <span class="n">agroups</span><span class="p">:</span>
<span class="k">if</span> <span class="n">agrp</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s">"</span><span class="si">%s</span><span class="s"> provides no packages for </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span>
<span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">agrp</span><span class="p">))</span>
<span class="k">continue</span>
<span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">agrp</span><span class="p">]</span><span class="o">.</span><span class="n">items</span><span class="p">()):</span>
<span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">vdict</span><span class="p">:</span>
<span class="n">vdict</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">vdict</span><span class="p">[</span><span class="n">key</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="k">return</span> <span class="n">vdict</span>
</div>
<div class="viewcode-block" id="Source.is_virtual_package"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.is_virtual_package">[docs]</a> <span class="k">def</span> <span class="nf">is_virtual_package</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span> <span class="c"># pylint: disable=W0613</span>
<span class="sd">""" Return True if a name is a virtual package (i.e., is a</span>
<span class="sd"> symbol provided by a real package), False otherwise.</span>
<span class="sd"> :param package: The name of the symbol, but see :ref:`pkg-objects`</span>
<span class="sd"> :type package: string</span>
<span class="sd"> :returns: bool</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="bp">False</span>
</div>
<div class="viewcode-block" id="Source.escape_url"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.escape_url">[docs]</a> <span class="k">def</span> <span class="nf">escape_url</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span><span class="p">):</span>
<span class="sd">""" Given a URL to a repository metadata file, return the full</span>
<span class="sd"> path to a file suitable for storing that file locally. This</span>
<span class="sd"> is acheived by replacing all forward slashes in the URL with</span>
<span class="sd"> ``@``.</span>
<span class="sd"> :param url: The URL to escape</span>
<span class="sd"> :type url: string</span>
<span class="sd"> :returns: string</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">basepath</span><span class="p">,</span> <span class="n">url</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'@'</span><span class="p">))</span>
</div>
<div class="viewcode-block" id="Source.read_files"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.read_files">[docs]</a> <span class="k">def</span> <span class="nf">read_files</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">""" Read and parse locally downloaded metadata files and</span>
<span class="sd"> populates</span>
<span class="sd"> :attr:`Bcfg2.Server.Plugins.Packages.Source.Source.pkgnames`. Should</span>
<span class="sd"> call</span>
<span class="sd"> :func:`Bcfg2.Server.Plugins.Packages.Source.Source.process_files`</span>
<span class="sd"> as its final step."""</span>
<span class="k">pass</span>
</div>
<div class="viewcode-block" id="Source.process_files"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.process_files">[docs]</a> <span class="k">def</span> <span class="nf">process_files</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dependencies</span><span class="p">,</span> <span class="n">provides</span><span class="p">):</span>
<span class="sd">""" Given dicts of depends and provides generated by</span>
<span class="sd"> :func:`read_files`, this generates :attr:`deps` and</span>
<span class="sd"> :attr:`provides` and calls :func:`save_state` to save the</span>
<span class="sd"> cached data to disk.</span>
<span class="sd"> Both arguments are dicts of dicts of lists. Keys are the</span>
<span class="sd"> arches of packages contained in this source; values are dicts</span>
<span class="sd"> whose keys are package names and values are lists of either</span>
<span class="sd"> dependencies for each package the symbols provided by each</span>
<span class="sd"> package.</span>
<span class="sd"> :param dependencies: A dict of dependencies found in the</span>
<span class="sd"> metadata for this source.</span>
<span class="sd"> :type dependencies: dict; see above.</span>
<span class="sd"> :param provides: A dict of symbols provided by packages in</span>
<span class="sd"> this repository.</span>
<span class="sd"> :type provides: dict; see above.</span>
<span class="sd"> """</span>
<span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="s">'global'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="s">'global'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">dependencies</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="n">barch</span><span class="p">]</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">]</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="k">for</span> <span class="n">pkgname</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span><span class="p">:</span>
<span class="n">pset</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">dependencies</span><span class="p">:</span>
<span class="k">if</span> <span class="n">pkgname</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">dependencies</span><span class="p">[</span><span class="n">barch</span><span class="p">]:</span>
<span class="n">dependencies</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">pset</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">dependencies</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]))</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">pset</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="s">'global'</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]</span> <span class="o">=</span> <span class="n">pset</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">dependencies</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]</span> <span class="o">=</span> <span class="n">dependencies</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]</span>
<span class="n">provided</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="k">for</span> <span class="n">bprovided</span> <span class="ow">in</span> <span class="nb">list</span><span class="p">(</span><span class="n">provides</span><span class="o">.</span><span class="n">values</span><span class="p">()):</span>
<span class="n">provided</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">bprovided</span><span class="p">))</span>
<span class="k">for</span> <span class="n">prov</span> <span class="ow">in</span> <span class="n">provided</span><span class="p">:</span>
<span class="n">prset</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">provides</span><span class="p">:</span>
<span class="k">if</span> <span class="n">prov</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">]:</span>
<span class="k">continue</span>
<span class="n">prset</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">]</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">prov</span><span class="p">,</span> <span class="p">())))</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">prset</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="s">'global'</span><span class="p">][</span><span class="n">prov</span><span class="p">]</span> <span class="o">=</span> <span class="n">prset</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">provides</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">prov</span><span class="p">]</span> <span class="o">=</span> <span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">]</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">prov</span><span class="p">,</span> <span class="p">())</span>
<span class="bp">self</span><span class="o">.</span><span class="n">save_state</span><span class="p">()</span>
</div>
<div class="viewcode-block" id="Source.unknown_filter"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.unknown_filter">[docs]</a> <span class="k">def</span> <span class="nf">unknown_filter</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span>
<span class="sd">""" A predicate that is used by :func:`filter_unknown` to</span>
<span class="sd"> filter packages from the results of</span>
<span class="sd"> :func:`Bcfg2.Server.Plugins.Packages.Collection.Collection.complete`</span>
<span class="sd"> that should not be shown to the end user (i.e., that are not</span>
<span class="sd"> truly unknown, but are rather packaging system artifacts). By</span>
<span class="sd"> default, excludes any package whose name starts with "choice"</span>
<span class="sd"> :param package: The name of a package that was unknown to the</span>
<span class="sd"> backend</span>
<span class="sd"> :type package: string</span>
<span class="sd"> :returns: bool</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="n">package</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">"choice"</span><span class="p">)</span>
</div>
<div class="viewcode-block" id="Source.filter_unknown"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.filter_unknown">[docs]</a> <span class="k">def</span> <span class="nf">filter_unknown</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">unknown</span><span class="p">):</span>
<span class="sd">""" After</span>
<span class="sd"> :func:`Bcfg2.Server.Plugins.Packages.Collection.Collection.complete`,</span>
<span class="sd"> filter out packages that appear in the list of unknown</span>
<span class="sd"> packages but should not be presented to the user.</span>
<span class="sd"> :attr:`unknown_filter` is called to assess whether or not a</span>
<span class="sd"> package is expected to be unknown.</span>
<span class="sd"> :param unknown: A set of unknown packages. The set should be</span>
<span class="sd"> modified in place.</span>
<span class="sd"> :type unknown: set of strings</span>
<span class="sd"> """</span>
<span class="n">unknown</span><span class="o">.</span><span class="n">difference_update</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">u</span> <span class="k">for</span> <span class="n">u</span> <span class="ow">in</span> <span class="n">unknown</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">unknown_filter</span><span class="p">(</span><span class="n">u</span><span class="p">)]))</span>
</div>
<div class="viewcode-block" id="Source.update"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.update">[docs]</a> <span class="k">def</span> <span class="nf">update</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">""" Download metadata from the upstream repository and cache</span>
<span class="sd"> it locally.</span>
<span class="sd"> :raises: ValueError - If any URL in :attr:`urls` is malformed</span>
<span class="sd"> :raises: OSError - If there is an error writing the local</span>
<span class="sd"> cache</span>
<span class="sd"> :raises: HTTPError - If there is an error fetching the remote</span>
<span class="sd"> data</span>
<span class="sd"> """</span>
<span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">urls</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s">"Packages: Updating </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">url</span><span class="p">)</span>
<span class="n">fname</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">escape_url</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="nb">open</span><span class="p">(</span><span class="n">fname</span><span class="p">,</span> <span class="s">'w'</span><span class="p">)</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">fetch_url</span><span class="p">(</span><span class="n">url</span><span class="p">))</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Bad url string </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">url</span><span class="p">)</span>
<span class="k">raise</span>
<span class="k">except</span> <span class="ne">OSError</span><span class="p">:</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Could not write data from </span><span class="si">%s</span><span class="s"> to "</span>
<span class="s">"local cache at </span><span class="si">%s</span><span class="s">: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">fname</span><span class="p">,</span> <span class="n">err</span><span class="p">))</span>
<span class="k">raise</span>
<span class="k">except</span> <span class="n">HTTPError</span><span class="p">:</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Failed to fetch url </span><span class="si">%s</span><span class="s">. HTTP "</span>
<span class="s">"response code=</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">err</span><span class="o">.</span><span class="n">code</span><span class="p">))</span>
<span class="k">raise</span>
</div>
<div class="viewcode-block" id="Source.applies"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.applies">[docs]</a> <span class="k">def</span> <span class="nf">applies</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span>
<span class="sd">""" Return true if this source applies to the given client,</span>
<span class="sd"> i.e., the client is in all necessary groups and</span>
<span class="sd"> :ref:`server-plugins-generators-packages-magic-groups`.</span>
<span class="sd"> :param metadata: The client metadata to check to see if this</span>
<span class="sd"> source applies</span>
<span class="sd"> :type metadata: Bcfg2.Server.Plugins.Metadata.ClientMetadata</span>
<span class="sd"> :returns: bool</span>
<span class="sd"> """</span>
<span class="c"># check base groups</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">magic_groups_match</span><span class="p">(</span><span class="n">metadata</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">False</span>
<span class="c"># check Group/Client tags from sources.xml</span>
<span class="k">for</span> <span class="n">condition</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="p">:</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">(</span><span class="n">metadata</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">False</span>
<span class="k">return</span> <span class="bp">True</span>
</div>
<div class="viewcode-block" id="Source.get_arches"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_arches">[docs]</a> <span class="k">def</span> <span class="nf">get_arches</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span>
<span class="sd">""" Get a list of architectures that the given client has and</span>
<span class="sd"> for which this source provides packages for. The return value</span>
<span class="sd"> will always include ``global``.</span>
<span class="sd"> :param metadata: The client metadata to get matching</span>
<span class="sd"> architectures for</span>
<span class="sd"> :type metadata: Bcfg2.Server.Plugins.Metadata.ClientMetadata</span>
<span class="sd"> :returns: list of strings</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="p">[</span><span class="s">'global'</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">a</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span> <span class="k">if</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span><span class="p">]</span>
</div>
<div class="viewcode-block" id="Source.get_deps"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_deps">[docs]</a> <span class="k">def</span> <span class="nf">get_deps</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span>
<span class="sd">""" Get a list of the dependencies of the given package.</span>
<span class="sd"> :param package: The name of the symbol</span>
<span class="sd"> :type package: string</span>
<span class="sd"> :returns: list of strings</span>
<span class="sd"> """</span>
<span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_arches</span><span class="p">(</span><span class="n">metadata</span><span class="p">):</span>
<span class="k">if</span> <span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="n">arch</span><span class="p">]:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="n">arch</span><span class="p">][</span><span class="n">package</span><span class="p">]</span>
<span class="k">return</span> <span class="p">[]</span>
</div>
<div class="viewcode-block" id="Source.get_provides"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_provides">[docs]</a> <span class="k">def</span> <span class="nf">get_provides</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span>
<span class="sd">""" Get a list of all symbols provided by the given package.</span>
<span class="sd"> :param package: The name of the package</span>
<span class="sd"> :type package: string</span>
<span class="sd"> :returns: list of strings</span>
<span class="sd"> """</span>
<span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_arches</span><span class="p">(</span><span class="n">metadata</span><span class="p">):</span>
<span class="k">if</span> <span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">arch</span><span class="p">]:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">arch</span><span class="p">][</span><span class="n">package</span><span class="p">]</span>
<span class="k">return</span> <span class="p">[]</span>
</div>
<div class="viewcode-block" id="Source.is_package"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.is_package">[docs]</a> <span class="k">def</span> <span class="nf">is_package</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span> <span class="c"># pylint: disable=W0613</span>
<span class="sd">""" Return True if a package is a package, False otherwise.</span>
<span class="sd"> :param package: The name of the package</span>
<span class="sd"> :type package: string</span>
<span class="sd"> :returns: bool</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="p">(</span><span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span> <span class="ow">and</span>
<span class="n">package</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">blacklist</span> <span class="ow">and</span>
<span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">whitelist</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">whitelist</span><span class="p">))</span>
</div>
<div class="viewcode-block" id="Source.get_group"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_group">[docs]</a> <span class="k">def</span> <span class="nf">get_group</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">group</span><span class="p">,</span> <span class="n">ptype</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span> <span class="c"># pylint: disable=W0613</span>
<span class="sd">""" Get the list of packages of the given type in a package</span>
<span class="sd"> group.</span>
<span class="sd"> :param group: The name of the group to query</span>
<span class="sd"> :type group: string</span>
<span class="sd"> :param ptype: The type of packages to get, for backends that</span>
<span class="sd"> support multiple package types in package groups</span>
<span class="sd"> (e.g., "recommended," "optional," etc.)</span>
<span class="sd"> :type ptype: string</span>
<span class="sd"> :returns: list of strings - package names</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="p">[]</span>
</div>
<div class="viewcode-block" id="Source.magic_groups_match"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.magic_groups_match">[docs]</a> <span class="k">def</span> <span class="nf">magic_groups_match</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span>
<span class="sd">""" Returns True if the client's</span>
<span class="sd"> :ref:`server-plugins-generators-packages-magic-groups` match</span>
<span class="sd"> the magic groups this source. Also returns True if magic</span>
<span class="sd"> groups are off in the configuration and the client's</span>
<span class="sd"> architecture matches (i.e., architecture groups are *always*</span>
<span class="sd"> checked).</span>
<span class="sd"> :returns: bool</span>
<span class="sd"> """</span>
<span class="n">found_arch</span> <span class="o">=</span> <span class="bp">False</span>
<span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span><span class="p">:</span>
<span class="k">if</span> <span class="n">arch</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span><span class="p">:</span>
<span class="n">found_arch</span> <span class="o">=</span> <span class="bp">True</span>
<span class="k">break</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">found_arch</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">False</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">setup</span><span class="o">.</span><span class="n">cfp</span><span class="o">.</span><span class="n">getboolean</span><span class="p">(</span><span class="s">"packages"</span><span class="p">,</span> <span class="s">"magic_groups"</span><span class="p">,</span>
<span class="n">default</span><span class="o">=</span><span class="bp">False</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">True</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">for</span> <span class="n">group</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">basegroups</span><span class="p">:</span>
<span class="k">if</span> <span class="n">group</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">True</span>
<span class="k">return</span> <span class="bp">False</span></div></div>
</pre></div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../../../../../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../../../../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../../../../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li><a href="../../../../../index.html">home</a> | </li>
<!--<li><a href="../../../../../search.html">search</a> | </li>-->
<li><a href="../../../../../help/index.html">help</a> | </li>
<li><a href="../../../../../contents.html">documentation </a> »</li>
<li><a href="../../../../index.html" >Module code</a> »</li>
<li><a href="../../Plugins.html" >Bcfg2.Server.Plugins</a> »</li>
<li><a href="../Packages.html" >Bcfg2.Server.Plugins.Packages</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright 2009-2013, Narayan Desai.
Last updated on Mar 20, 2013.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
</div>
</body>
</html>
distrib
>
Fedora
>
17
>
i386
>
media
>
updates
>
by-pkgid
>
b50d8ee6d7871fcc13c0677a9364ed59
>
files
>
55
