<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Bcfg2.Server.Plugins.Packages.Source — Bcfg2 1.3.0 documentation</title> <link rel="stylesheet" href="../../../../../_static/default.css" type="text/css" /> <link rel="stylesheet" href="../../../../../_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: '../../../../../', VERSION: '1.3.0', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="../../../../../_static/jquery.js"></script> <script type="text/javascript" src="../../../../../_static/underscore.js"></script> <script type="text/javascript" src="../../../../../_static/doctools.js"></script> <script type="text/javascript" src="../../../../../_static/sidebar.js"></script> <link rel="shortcut icon" href="../../../../../_static/favicon.ico"/> <link rel="top" title="Bcfg2 1.3.0 documentation" href="../../../../../index.html" /> <link rel="up" title="Bcfg2.Server.Plugins.Packages" href="../Packages.html" /> <link rel="stylesheet" href="../../../../../_static/bcfg2.css" type=""/> </head> <body> <div style="text-align: left; padding: 10px 10px 15px 15px"> <a href="../../../../../index.html"><img src="../../../../../_static/bcfg2_logo.png" border="0" alt="sampledoc"/></a> </div> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="../../../../../genindex.html" title="General Index" accesskey="I">index</a></li> <li class="right" > <a href="../../../../../py-modindex.html" title="Python Module Index" >modules</a> |</li> <li><a href="../../../../../index.html">home</a> | </li> <!--<li><a href="../../../../../search.html">search</a> | </li>--> <li><a href="../../../../../help/index.html">help</a> | </li> <li><a href="../../../../../contents.html">documentation </a> »</li> <li><a href="../../../../index.html" >Module code</a> »</li> <li><a href="../../Plugins.html" >Bcfg2.Server.Plugins</a> »</li> <li><a href="../Packages.html" accesskey="U">Bcfg2.Server.Plugins.Packages</a> »</li> </ul> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <h1>Source code for Bcfg2.Server.Plugins.Packages.Source</h1><div class="highlight"><pre> <span class="sd">""" ``Source`` objects represent a single <Source> tag in</span> <span class="sd">``sources.xml``. Note that a single Source tag can itself describe</span> <span class="sd">multiple repositories (if it uses the "url" attribute instead of</span> <span class="sd">"rawurl"), and so can the ``Source`` object. This can be the source</span> <span class="sd">(har har) of some confusion. See</span> <span class="sd">:func:`Bcfg2.Server.Plugins.Packages.Collection.Collection.sourcelist`</span> <span class="sd">for the proper way to get all repos from a ``Source`` object.</span> <span class="sd">Source objects are aggregated into</span> <span class="sd">:class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`</span> <span class="sd">objects, which are actually called by</span> <span class="sd">:class:`Bcfg2.Server.Plugins.Packages.Packages`. This way a more</span> <span class="sd">advanced subclass can query repositories in aggregate rather than</span> <span class="sd">individually, which may give faster or more accurate results.</span> <span class="sd">The base ``Source`` object must be subclassed to handle each</span> <span class="sd">repository type. How you subclass ``Source`` will depend on how you</span> <span class="sd">subclassed</span> <span class="sd">:class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`; see</span> <span class="sd">:mod:`Bcfg2.Server.Plugins.Packages.Collection` for more details on</span> <span class="sd">different methods for doing that.</span> <span class="sd">If you are using the stock (or a near-stock)</span> <span class="sd">:class:`Bcfg2.Server.Plugins.Packages.Collection.Collection` object,</span> <span class="sd">then you will need to implement the following methods and attributes</span> <span class="sd">in your ``Source`` subclass:</span> <span class="sd">* :func:`Source.urls`</span> <span class="sd">* :func:`Source.read_files`</span> <span class="sd">* :attr:`Source.basegroups`</span> <span class="sd">Additionally, you may want to consider overriding the following</span> <span class="sd">methods and attributes:</span> <span class="sd">* :func:`Source.is_virtual_package`</span> <span class="sd">* :func:`Source.get_group`</span> <span class="sd">* :attr:`Source.unknown_filter`</span> <span class="sd">* :attr:`Source.load_state`</span> <span class="sd">* :attr:`Source.save_state`</span> <span class="sd">For an example of this kind of ``Source`` object, see</span> <span class="sd">:mod:`Bcfg2.Server.Plugins.Packages.Apt`.</span> <span class="sd">If you are overriding the ``Collection`` object in more depth, then</span> <span class="sd">you have more leeway in what you might want to override or implement</span> <span class="sd">in your ``Source`` subclass. For an example of this kind of</span> <span class="sd">``Source`` object, see :mod:`Bcfg2.Server.Plugins.Packages.Yum`.</span> <span class="sd">"""</span> <span class="kn">import</span> <span class="nn">os</span> <span class="kn">import</span> <span class="nn">re</span> <span class="kn">import</span> <span class="nn">sys</span> <span class="kn">import</span> <span class="nn">Bcfg2.Server.Plugin</span> <span class="kn">from</span> <span class="nn">Bcfg2.Compat</span> <span class="kn">import</span> <span class="n">HTTPError</span><span class="p">,</span> <span class="n">HTTPBasicAuthHandler</span><span class="p">,</span> \ <span class="n">HTTPPasswordMgrWithDefaultRealm</span><span class="p">,</span> <span class="n">install_opener</span><span class="p">,</span> <span class="n">build_opener</span><span class="p">,</span> \ <span class="n">urlopen</span><span class="p">,</span> <span class="n">cPickle</span><span class="p">,</span> <span class="n">md5</span> <div class="viewcode-block" id="fetch_url"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.fetch_url">[docs]</a><span class="k">def</span> <span class="nf">fetch_url</span><span class="p">(</span><span class="n">url</span><span class="p">):</span> <span class="sd">""" Return the content of the given URL.</span> <span class="sd"> :param url: The URL to fetch content from.</span> <span class="sd"> :type url: string</span> <span class="sd"> :raises: ValueError - Malformed URL</span> <span class="sd"> :raises: URLError - Failure fetching URL</span> <span class="sd"> :returns: string - the content of the page at the given URL """</span> <span class="k">if</span> <span class="s">'@'</span> <span class="ow">in</span> <span class="n">url</span><span class="p">:</span> <span class="n">mobj</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s">'(\w+://)([^:]+):([^@]+)@(.*)$'</span><span class="p">,</span> <span class="n">url</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">mobj</span><span class="p">:</span> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">"Invalid URL"</span><span class="p">)</span> <span class="n">user</span> <span class="o">=</span> <span class="n">mobj</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="n">passwd</span> <span class="o">=</span> <span class="n">mobj</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="n">url</span> <span class="o">=</span> <span class="n">mobj</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="o">+</span> <span class="n">mobj</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span> <span class="n">auth</span> <span class="o">=</span> <span class="n">HTTPBasicAuthHandler</span><span class="p">(</span><span class="n">HTTPPasswordMgrWithDefaultRealm</span><span class="p">())</span> <span class="n">auth</span><span class="o">.</span><span class="n">add_password</span><span class="p">(</span><span class="bp">None</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="n">user</span><span class="p">,</span> <span class="n">passwd</span><span class="p">)</span> <span class="n">install_opener</span><span class="p">(</span><span class="n">build_opener</span><span class="p">(</span><span class="n">auth</span><span class="p">))</span> <span class="k">return</span> <span class="n">urlopen</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> </div> <div class="viewcode-block" id="SourceInitError"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.SourceInitError">[docs]</a><span class="k">class</span> <span class="nc">SourceInitError</span><span class="p">(</span><span class="ne">Exception</span><span class="p">):</span> <span class="sd">""" Raised when a :class:`Source` object fails instantiation. """</span> <span class="k">pass</span> <span class="c">#: A regular expression used to determine the base name of a repo from</span> <span class="c">#: its URL. This is used when generating repo configs and by</span> <span class="c">#: :func:`Source.get_repo_name`. It handles `Pulp</span> <span class="c">#: <http://www.pulpproject.org/>`_ and `mrepo</span> <span class="c">#: <http://dag.wieers.com/home-made/mrepo/>`_ repositories specially,</span> <span class="c">#: and otherwise grabs the last component of the URL (as delimited by</span> <span class="c">#: slashes).</span></div> <span class="n">REPO_RE</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="s">r'(?:pulp/repos/|/RPMS\.|/)([^/]+)/?$'</span><span class="p">)</span> <div class="viewcode-block" id="Source"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source">[docs]</a><span class="k">class</span> <span class="nc">Source</span><span class="p">(</span><span class="n">Bcfg2</span><span class="o">.</span><span class="n">Server</span><span class="o">.</span><span class="n">Plugin</span><span class="o">.</span><span class="n">Debuggable</span><span class="p">):</span> <span class="c"># pylint: disable=R0902</span> <span class="sd">""" ``Source`` objects represent a single <Source> tag in</span> <span class="sd"> ``sources.xml``. Note that a single Source tag can itself</span> <span class="sd"> describe multiple repositories (if it uses the "url" attribute</span> <span class="sd"> instead of "rawurl"), and so can the ``Source`` object.</span> <span class="sd"> Note that a number of the attributes of this object may be more or</span> <span class="sd"> less specific to one backend (e.g., :attr:`essentialpkgs`,</span> <span class="sd"> :attr:`recommended`, :attr:`gpgkeys`, but they are included in the</span> <span class="sd"> superclass to make the parsing of sources from XML more</span> <span class="sd"> consistent, and to make it trivial for other backends to support</span> <span class="sd"> those features.</span> <span class="sd"> """</span> <span class="c">#: The list of</span> <span class="c">#: :ref:`server-plugins-generators-packages-magic-groups` that</span> <span class="c">#: make sources of this type available to clients.</span> <span class="n">basegroups</span> <span class="o">=</span> <span class="p">[]</span> <span class="c">#: The Package type handled by this Source class. The ``type``</span> <span class="c">#: attribute of Package entries will be set to the value ``ptype``</span> <span class="c">#: when they are handled by :mod:`Bcfg2.Server.Plugins.Packages`.</span> <span class="n">ptype</span> <span class="o">=</span> <span class="bp">None</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">basepath</span><span class="p">,</span> <span class="n">xsource</span><span class="p">,</span> <span class="n">setup</span><span class="p">):</span> <span class="c"># pylint: disable=R0912</span> <span class="sd">"""</span> <span class="sd"> :param basepath: The base filesystem path under which cache</span> <span class="sd"> data for this source should be stored</span> <span class="sd"> :type basepath: string</span> <span class="sd"> :param xsource: The XML tag that describes this source</span> <span class="sd"> :type source: lxml.etree._Element</span> <span class="sd"> :param setup: A Bcfg2 options dict</span> <span class="sd"> :type setup: dict</span> <span class="sd"> :raises: :class:`Bcfg2.Server.Plugins.Packages.Source.SourceInitError`</span> <span class="sd"> """</span> <span class="n">Bcfg2</span><span class="o">.</span><span class="n">Server</span><span class="o">.</span><span class="n">Plugin</span><span class="o">.</span><span class="n">Debuggable</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="c">#: The base filesystem path under which cache data for this</span> <span class="c">#: source should be stored</span> <span class="bp">self</span><span class="o">.</span><span class="n">basepath</span> <span class="o">=</span> <span class="n">basepath</span> <span class="c">#: The XML tag that describes this source</span> <span class="bp">self</span><span class="o">.</span><span class="n">xsource</span> <span class="o">=</span> <span class="n">xsource</span> <span class="c">#: A Bcfg2 options dict</span> <span class="bp">self</span><span class="o">.</span><span class="n">setup</span> <span class="o">=</span> <span class="n">setup</span> <span class="c">#: A set of package names that are deemed "essential" by this</span> <span class="c">#: source</span> <span class="bp">self</span><span class="o">.</span><span class="n">essentialpkgs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span> <span class="c">#: A list of the text of all 'Component' attributes of this</span> <span class="c">#: source from XML</span> <span class="bp">self</span><span class="o">.</span><span class="n">components</span> <span class="o">=</span> <span class="p">[</span><span class="n">item</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'Component'</span><span class="p">)]</span> <span class="c">#: A list of the arches supported by this source</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span> <span class="o">=</span> <span class="p">[</span><span class="n">item</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'Arch'</span><span class="p">)]</span> <span class="c">#: A list of the the names of packages that are blacklisted</span> <span class="c">#: from this source</span> <span class="bp">self</span><span class="o">.</span><span class="n">blacklist</span> <span class="o">=</span> <span class="p">[</span><span class="n">item</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'Blacklist'</span><span class="p">)]</span> <span class="c">#: A list of the the names of packages that are whitelisted in</span> <span class="c">#: this source</span> <span class="bp">self</span><span class="o">.</span><span class="n">whitelist</span> <span class="o">=</span> <span class="p">[</span><span class="n">item</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">'Whitelist'</span><span class="p">)]</span> <span class="c">#: A dict of repository options that will be included in the</span> <span class="c">#: configuration generated on the server side (if such is</span> <span class="c">#: applicable; most backends do not generate any sort of</span> <span class="c">#: repository configuration on the Bcfg2 server)</span> <span class="bp">self</span><span class="o">.</span><span class="n">server_options</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="c">#: A dict of repository options that will be included in the</span> <span class="c">#: configuration generated for the client (if that is</span> <span class="c">#: supported by the backend)</span> <span class="bp">self</span><span class="o">.</span><span class="n">client_options</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="n">opts</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">"Options"</span><span class="p">)</span> <span class="k">for</span> <span class="n">el</span> <span class="ow">in</span> <span class="n">opts</span><span class="p">:</span> <span class="n">repoopts</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">([(</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">el</span><span class="o">.</span><span class="n">attrib</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">k</span> <span class="o">!=</span> <span class="s">"clientonly"</span> <span class="ow">and</span> <span class="n">k</span> <span class="o">!=</span> <span class="s">"serveronly"</span><span class="p">])</span> <span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"clientonly"</span><span class="p">,</span> <span class="s">"false"</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">"false"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">server_options</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">repoopts</span><span class="p">)</span> <span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"serveronly"</span><span class="p">,</span> <span class="s">"false"</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">"false"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">client_options</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">repoopts</span><span class="p">)</span> <span class="c">#: A list of URLs to GPG keys that apply to this source</span> <span class="bp">self</span><span class="o">.</span><span class="n">gpgkeys</span> <span class="o">=</span> <span class="p">[</span><span class="n">el</span><span class="o">.</span><span class="n">text</span> <span class="k">for</span> <span class="n">el</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span><span class="s">"GPGKey"</span><span class="p">)]</span> <span class="c">#: Whether or not to include essential packages from this source</span> <span class="bp">self</span><span class="o">.</span><span class="n">essential</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'essential'</span><span class="p">,</span> <span class="s">'true'</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">'true'</span> <span class="c">#: Whether or not to include recommended packages from this source</span> <span class="bp">self</span><span class="o">.</span><span class="n">recommended</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'recommended'</span><span class="p">,</span> <span class="s">'false'</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">'true'</span> <span class="c">#: The "rawurl" attribute from :attr:`xsource`, if applicable.</span> <span class="c">#: A trailing slash is automatically appended to this if there</span> <span class="c">#: wasn't one already present.</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'rawurl'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s">"/"</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span> <span class="o">+=</span> <span class="s">"/"</span> <span class="c">#: The "url" attribute from :attr:`xsource`, if applicable. A</span> <span class="c">#: trailing slash is automatically appended to this if there</span> <span class="c">#: wasn't one already present.</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'url'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s">"/"</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">+=</span> <span class="s">"/"</span> <span class="c">#: The "version" attribute from :attr:`xsource`</span> <span class="bp">self</span><span class="o">.</span><span class="n">version</span> <span class="o">=</span> <span class="n">xsource</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'version'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span> <span class="c">#: A list of predicates that are used to determine if this</span> <span class="c">#: source applies to a given</span> <span class="c">#: :class:`Bcfg2.Server.Plugins.Metadata.ClientMetadata`</span> <span class="c">#: object.</span> <span class="bp">self</span><span class="o">.</span><span class="n">conditions</span> <span class="o">=</span> <span class="p">[]</span> <span class="c">#: Formerly, :ref:`server-plugins-generators-packages` only</span> <span class="c">#: supported applying package sources to groups; that is, they</span> <span class="c">#: could not be assigned by more complicated logic like</span> <span class="c">#: per-client repositories and group or client negation. This</span> <span class="c">#: attribute attempts to provide for some limited backwards</span> <span class="c">#: compat with older code that relies on this.</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">for</span> <span class="n">el</span> <span class="ow">in</span> <span class="n">xsource</span><span class="o">.</span><span class="n">iterancestors</span><span class="p">():</span> <span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">tag</span> <span class="o">==</span> <span class="s">"Group"</span><span class="p">:</span> <span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"negate"</span><span class="p">,</span> <span class="s">"false"</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">"true"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="k">lambda</span> <span class="n">m</span><span class="p">,</span> <span class="n">el</span><span class="o">=</span><span class="n">el</span><span class="p">:</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">m</span><span class="o">.</span><span class="n">groups</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">))</span> <span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="k">lambda</span> <span class="n">m</span><span class="p">,</span> <span class="n">el</span><span class="o">=</span><span class="n">el</span><span class="p">:</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span> <span class="ow">in</span> <span class="n">m</span><span class="o">.</span><span class="n">groups</span><span class="p">)</span> <span class="k">elif</span> <span class="n">el</span><span class="o">.</span><span class="n">tag</span> <span class="o">==</span> <span class="s">"Client"</span><span class="p">:</span> <span class="k">if</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"negate"</span><span class="p">,</span> <span class="s">"false"</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">"true"</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="k">lambda</span> <span class="n">m</span><span class="p">,</span> <span class="n">el</span><span class="o">=</span><span class="n">el</span><span class="p">:</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span> <span class="o">!=</span> <span class="n">m</span><span class="o">.</span><span class="n">hostname</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="k">lambda</span> <span class="n">m</span><span class="p">,</span> <span class="n">el</span><span class="o">=</span><span class="n">el</span><span class="p">:</span> <span class="n">el</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">)</span> <span class="o">==</span> <span class="n">m</span><span class="o">.</span><span class="n">hostname</span><span class="p">)</span> <span class="c">#: A set of all package names in this source. This will not</span> <span class="c">#: necessarily be populated, particularly by backends that</span> <span class="c">#: reimplement large portions of</span> <span class="c">#: :class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`</span> <span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span> <span class="c">#: A dict of ``<package name>`` -> ``<list of dependencies>``.</span> <span class="c">#: This will not necessarily be populated, particularly by</span> <span class="c">#: backends that reimplement large portions of</span> <span class="c">#: :class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="c">#: A dict of ``<package name>`` -> ``<list of provided</span> <span class="c">#: symbols>``. This will not necessarily be populated,</span> <span class="c">#: particularly by backends that reimplement large portions of</span> <span class="c">#: :class:`Bcfg2.Server.Plugins.Packages.Collection.Collection`</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="c">#: The file (or directory) used for this source's cache data</span> <span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">basepath</span><span class="p">,</span> <span class="s">"cache-</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">cachekey</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">:</span> <span class="n">baseurl</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">+</span> <span class="s">"</span><span class="si">%(version)s</span><span class="s">/</span><span class="si">%(component)s</span><span class="s">/</span><span class="si">%(arch)s</span><span class="s">/"</span> <span class="k">else</span><span class="p">:</span> <span class="n">baseurl</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span> <span class="c">#: A list of dicts, each of which describes the URL to one</span> <span class="c">#: repository contained in this source. Each dict contains</span> <span class="c">#: the following keys:</span> <span class="c">#:</span> <span class="c">#: * ``version``: The version of the repo (``None`` for</span> <span class="c">#: ``rawurl`` repos)</span> <span class="c">#: * ``component``: The component use to form this URL</span> <span class="c">#: (``None`` for ``rawurl`` repos)</span> <span class="c">#: * ``arch``: The architecture of this repo</span> <span class="c">#: * ``baseurl``: Either the ``rawurl`` attribute, or the</span> <span class="c">#: format string built from the ``url`` attribute</span> <span class="c">#: * ``url``: The actual URL to the repository</span> <span class="bp">self</span><span class="o">.</span><span class="n">url_map</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span><span class="p">:</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">:</span> <span class="n">usettings</span> <span class="o">=</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="n">version</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">version</span><span class="p">,</span> <span class="n">component</span><span class="o">=</span><span class="n">comp</span><span class="p">,</span> <span class="n">arch</span><span class="o">=</span><span class="n">arch</span><span class="p">)</span> <span class="k">for</span> <span class="n">comp</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">components</span><span class="p">]</span> <span class="k">else</span><span class="p">:</span> <span class="c"># rawurl given</span> <span class="n">usettings</span> <span class="o">=</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="n">version</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">version</span><span class="p">,</span> <span class="n">component</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">arch</span><span class="o">=</span><span class="n">arch</span><span class="p">)]</span> <span class="k">for</span> <span class="n">setting</span> <span class="ow">in</span> <span class="n">usettings</span><span class="p">:</span> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">:</span> <span class="n">setting</span><span class="p">[</span><span class="s">'baseurl'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="k">else</span><span class="p">:</span> <span class="n">setting</span><span class="p">[</span><span class="s">'baseurl'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span> <span class="n">setting</span><span class="p">[</span><span class="s">'url'</span><span class="p">]</span> <span class="o">=</span> <span class="n">baseurl</span> <span class="o">%</span> <span class="n">setting</span> <span class="bp">self</span><span class="o">.</span><span class="n">url_map</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">usettings</span><span class="p">)</span> <span class="nd">@property</span> <div class="viewcode-block" id="Source.cachekey"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.cachekey">[docs]</a> <span class="k">def</span> <span class="nf">cachekey</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="sd">""" A unique key for this source that will be used to generate</span> <span class="sd"> :attr:`cachefile` and other cache paths """</span> <span class="k">return</span> <span class="n">md5</span><span class="p">(</span><span class="n">cPickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">([</span><span class="bp">self</span><span class="o">.</span><span class="n">version</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">components</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span><span class="p">]))</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">()</span> </div> <div class="viewcode-block" id="Source.get_relevant_groups"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_relevant_groups">[docs]</a> <span class="k">def</span> <span class="nf">get_relevant_groups</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span> <span class="sd">""" Get all groups that might be relevant to determining which</span> <span class="sd"> sources apply to this collection's client.</span> <span class="sd"> :return: list of strings - group names</span> <span class="sd"> """</span> <span class="k">return</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">g</span> <span class="k">for</span> <span class="n">g</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span> <span class="k">if</span> <span class="p">(</span><span class="n">g</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">basegroups</span> <span class="ow">or</span> <span class="n">g</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span> <span class="ow">or</span> <span class="n">g</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span><span class="p">)])))</span> </div> <div class="viewcode-block" id="Source.load_state"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.load_state">[docs]</a> <span class="k">def</span> <span class="nf">load_state</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="sd">""" Load saved state from :attr:`cachefile`. If caching and</span> <span class="sd"> state is handled by the package library, then this function</span> <span class="sd"> does not need to be implemented.</span> <span class="sd"> :raises: OSError - If the saved data cannot be read</span> <span class="sd"> :raises: cPickle.UnpicklingError - If the saved data is corrupt """</span> <span class="n">data</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span><span class="p">)</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">essentialpkgs</span><span class="p">)</span> <span class="o">=</span> <span class="n">cPickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> </div> <div class="viewcode-block" id="Source.save_state"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.save_state">[docs]</a> <span class="k">def</span> <span class="nf">save_state</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="sd">""" Save state to :attr:`cachefile`. If caching and</span> <span class="sd"> state is handled by the package library, then this function</span> <span class="sd"> does not need to be implemented. """</span> <span class="n">cache</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span> <span class="n">cPickle</span><span class="o">.</span><span class="n">dump</span><span class="p">((</span><span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">essentialpkgs</span><span class="p">),</span> <span class="n">cache</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> <span class="n">cache</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> </div> <span class="nd">@Bcfg2.Server.Plugin.track_statistics</span><span class="p">()</span> <div class="viewcode-block" id="Source.setup_data"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.setup_data">[docs]</a> <span class="k">def</span> <span class="nf">setup_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">force_update</span><span class="o">=</span><span class="bp">False</span><span class="p">):</span> <span class="sd">""" Perform all data fetching and setup tasks. For most</span> <span class="sd"> backends, this involves downloading all metadata from the</span> <span class="sd"> repository, parsing it, and caching the parsed data locally.</span> <span class="sd"> The order of operations is:</span> <span class="sd"> #. Call :func:`load_state` to try to load data from the local</span> <span class="sd"> cache.</span> <span class="sd"> #. If that fails, call :func:`read_files` to read and parse</span> <span class="sd"> the locally downloaded metadata files.</span> <span class="sd"> #. If that fails, call :func:`update` to fetch the metadata,</span> <span class="sd"> then :func:`read_files` to parse it.</span> <span class="sd"> Obviously with a backend that leverages repo access libraries</span> <span class="sd"> to avoid downloading all metadata, many of the functions</span> <span class="sd"> called by ``setup_data`` can be no-ops (or nearly so).</span> <span class="sd"> :param force_update: Ignore all locally cached and downloaded</span> <span class="sd"> data and fetch the metadata anew from the</span> <span class="sd"> upstream repository.</span> <span class="sd"> :type force_update: bool</span> <span class="sd"> """</span> <span class="c"># pylint: disable=W0702</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">force_update</span><span class="p">:</span> <span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span><span class="p">):</span> <span class="k">try</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">load_state</span><span class="p">()</span> <span class="k">except</span><span class="p">:</span> <span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Cachefile </span><span class="si">%s</span><span class="s"> load failed: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cachefile</span><span class="p">,</span> <span class="n">err</span><span class="p">))</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Falling back to file read"</span><span class="p">)</span> <span class="k">try</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">read_files</span><span class="p">()</span> <span class="k">except</span><span class="p">:</span> <span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: File read failed: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">err</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Falling back to file download"</span><span class="p">)</span> <span class="n">force_update</span> <span class="o">=</span> <span class="bp">True</span> <span class="k">else</span><span class="p">:</span> <span class="n">force_update</span> <span class="o">=</span> <span class="bp">True</span> <span class="k">if</span> <span class="n">force_update</span><span class="p">:</span> <span class="k">try</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">update</span><span class="p">()</span> <span class="bp">self</span><span class="o">.</span><span class="n">read_files</span><span class="p">()</span> <span class="k">except</span><span class="p">:</span> <span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Failed to load data for </span><span class="si">%s</span><span class="s">: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">err</span><span class="p">))</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Some Packages will be missing"</span><span class="p">)</span> <span class="c"># pylint: enable=W0702</span> </div> <div class="viewcode-block" id="Source.get_repo_name"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_repo_name">[docs]</a> <span class="k">def</span> <span class="nf">get_repo_name</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url_map</span><span class="p">):</span> <span class="sd">""" Try to find a sensible name for a repository. Since</span> <span class="sd"> ``sources.xml`` doesn't provide for repository names, we have</span> <span class="sd"> to try to guess at the names when generating config files or</span> <span class="sd"> doing other operations that require repository names. This</span> <span class="sd"> function tries several approaches:</span> <span class="sd"> #. First, if the map contains a ``component`` key, use that as</span> <span class="sd"> the name.</span> <span class="sd"> #. If not, then try to match the repository URL against</span> <span class="sd"> :attr:`Bcfg2.Server.Plugins.Packages.Source.REPO_RE`. If</span> <span class="sd"> that succeeds, use the first matched group; additionally,</span> <span class="sd"> if the Source tag that describes this repo is contained in</span> <span class="sd"> a ``<Group>`` tag, prepend that to the name.</span> <span class="sd"> #. If :attr:`Bcfg2.Server.Plugins.Packages.Source.REPO_RE`</span> <span class="sd"> does not match the repository, and the Source tag that</span> <span class="sd"> describes this repo is contained in a ``<Group>`` tag, use</span> <span class="sd"> the name of the group.</span> <span class="sd"> #. Failing that, use the full URL to this repository, with the</span> <span class="sd"> protocol and trailing slash stripped off if possible.</span> <span class="sd"> Once that is done, all characters disallowed in yum source</span> <span class="sd"> names are replaced by dashes. See below for the exact regex.</span> <span class="sd"> The yum rules are used here because they are so restrictive.</span> <span class="sd"> ``get_repo_name`` is **not** guaranteed to return a unique</span> <span class="sd"> name. If you require a unique name, then you will need to</span> <span class="sd"> generate all repo names and make them unique through the</span> <span class="sd"> approach of your choice, e.g., appending numbers to non-unique</span> <span class="sd"> repository names. See</span> <span class="sd"> :func:`Bcfg2.Server.Plugins.Packages.Yum.Source.get_repo_name`</span> <span class="sd"> for an example.</span> <span class="sd"> :param url_map: A single :attr:`url_map` dict, i.e., any</span> <span class="sd"> single element of :attr:`url_map`.</span> <span class="sd"> :type url_map: dict</span> <span class="sd"> :returns: string - the name of the repository.</span> <span class="sd"> """</span> <span class="k">if</span> <span class="n">url_map</span><span class="p">[</span><span class="s">'component'</span><span class="p">]:</span> <span class="n">rname</span> <span class="o">=</span> <span class="n">url_map</span><span class="p">[</span><span class="s">'component'</span><span class="p">]</span> <span class="k">else</span><span class="p">:</span> <span class="n">match</span> <span class="o">=</span> <span class="n">REPO_RE</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">url_map</span><span class="p">[</span><span class="s">'url'</span><span class="p">])</span> <span class="k">if</span> <span class="n">match</span><span class="p">:</span> <span class="n">rname</span> <span class="o">=</span> <span class="n">match</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="p">:</span> <span class="n">rname</span> <span class="o">=</span> <span class="s">"</span><span class="si">%s</span><span class="s">-</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">rname</span><span class="p">)</span> <span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="p">:</span> <span class="n">rname</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">groups</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">else</span><span class="p">:</span> <span class="c"># a global source with no reasonable name. Try to</span> <span class="c"># strip off the protocol and trailing slash.</span> <span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="s">r'^[A-z]://(.*?)/?'</span><span class="p">,</span> <span class="n">url_map</span><span class="p">[</span><span class="s">'url'</span><span class="p">])</span> <span class="k">if</span> <span class="n">match</span><span class="p">:</span> <span class="n">rname</span> <span class="o">=</span> <span class="n">match</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="c"># what kind of crazy url is this? I give up!</span> <span class="c"># just use the full url and let the regex below</span> <span class="c"># make it even uglier.</span> <span class="n">rname</span> <span class="o">=</span> <span class="n">url_map</span><span class="p">[</span><span class="s">'url'</span><span class="p">]</span> <span class="c"># see yum/__init__.py in the yum source, lines 441-449, for</span> <span class="c"># the source of this regex. yum doesn't like anything but</span> <span class="c"># string.ascii_letters, string.digits, and [-_.:]. There</span> <span class="c"># doesn't seem to be a reason for this, because yum.</span> <span class="k">return</span> <span class="n">re</span><span class="o">.</span><span class="n">sub</span><span class="p">(</span><span class="s">r'[^A-Za-z0-9-_.:]'</span><span class="p">,</span> <span class="s">'-'</span><span class="p">,</span> <span class="n">rname</span><span class="p">)</span> </div> <span class="k">def</span> <span class="nf">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">:</span> <span class="k">return</span> <span class="s">"</span><span class="si">%s</span><span class="s"> at </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">rawurl</span><span class="p">)</span> <span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">:</span> <span class="k">return</span> <span class="s">"</span><span class="si">%s</span><span class="s"> at </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">url</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span> <span class="nd">@property</span> <div class="viewcode-block" id="Source.urls"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.urls">[docs]</a> <span class="k">def</span> <span class="nf">urls</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="sd">""" A list of URLs to the base metadata file for each</span> <span class="sd"> repository described by this source. """</span> <span class="k">return</span> <span class="p">[]</span> </div> <span class="nd">@property</span> <div class="viewcode-block" id="Source.files"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.files">[docs]</a> <span class="k">def</span> <span class="nf">files</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="sd">""" A list of files stored in the local cache by this backend.</span> <span class="sd"> """</span> <span class="k">return</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">escape_url</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">urls</span><span class="p">]</span> </div> <div class="viewcode-block" id="Source.get_vpkgs"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_vpkgs">[docs]</a> <span class="k">def</span> <span class="nf">get_vpkgs</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span> <span class="sd">""" Get a list of all virtual packages provided by all sources.</span> <span class="sd"> :returns: list of strings</span> <span class="sd"> """</span> <span class="n">agroups</span> <span class="o">=</span> <span class="p">[</span><span class="s">'global'</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">a</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span> <span class="k">if</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span><span class="p">]</span> <span class="n">vdict</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="k">for</span> <span class="n">agrp</span> <span class="ow">in</span> <span class="n">agroups</span><span class="p">:</span> <span class="k">if</span> <span class="n">agrp</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s">"</span><span class="si">%s</span><span class="s"> provides no packages for </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">agrp</span><span class="p">))</span> <span class="k">continue</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">agrp</span><span class="p">]</span><span class="o">.</span><span class="n">items</span><span class="p">()):</span> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">vdict</span><span class="p">:</span> <span class="n">vdict</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="n">vdict</span><span class="p">[</span><span class="n">key</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="k">return</span> <span class="n">vdict</span> </div> <div class="viewcode-block" id="Source.is_virtual_package"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.is_virtual_package">[docs]</a> <span class="k">def</span> <span class="nf">is_virtual_package</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span> <span class="c"># pylint: disable=W0613</span> <span class="sd">""" Return True if a name is a virtual package (i.e., is a</span> <span class="sd"> symbol provided by a real package), False otherwise.</span> <span class="sd"> :param package: The name of the symbol, but see :ref:`pkg-objects`</span> <span class="sd"> :type package: string</span> <span class="sd"> :returns: bool</span> <span class="sd"> """</span> <span class="k">return</span> <span class="bp">False</span> </div> <div class="viewcode-block" id="Source.escape_url"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.escape_url">[docs]</a> <span class="k">def</span> <span class="nf">escape_url</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span><span class="p">):</span> <span class="sd">""" Given a URL to a repository metadata file, return the full</span> <span class="sd"> path to a file suitable for storing that file locally. This</span> <span class="sd"> is acheived by replacing all forward slashes in the URL with</span> <span class="sd"> ``@``.</span> <span class="sd"> :param url: The URL to escape</span> <span class="sd"> :type url: string</span> <span class="sd"> :returns: string</span> <span class="sd"> """</span> <span class="k">return</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">basepath</span><span class="p">,</span> <span class="n">url</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'@'</span><span class="p">))</span> </div> <div class="viewcode-block" id="Source.read_files"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.read_files">[docs]</a> <span class="k">def</span> <span class="nf">read_files</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="sd">""" Read and parse locally downloaded metadata files and</span> <span class="sd"> populates</span> <span class="sd"> :attr:`Bcfg2.Server.Plugins.Packages.Source.Source.pkgnames`. Should</span> <span class="sd"> call</span> <span class="sd"> :func:`Bcfg2.Server.Plugins.Packages.Source.Source.process_files`</span> <span class="sd"> as its final step."""</span> <span class="k">pass</span> </div> <div class="viewcode-block" id="Source.process_files"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.process_files">[docs]</a> <span class="k">def</span> <span class="nf">process_files</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dependencies</span><span class="p">,</span> <span class="n">provides</span><span class="p">):</span> <span class="sd">""" Given dicts of depends and provides generated by</span> <span class="sd"> :func:`read_files`, this generates :attr:`deps` and</span> <span class="sd"> :attr:`provides` and calls :func:`save_state` to save the</span> <span class="sd"> cached data to disk.</span> <span class="sd"> Both arguments are dicts of dicts of lists. Keys are the</span> <span class="sd"> arches of packages contained in this source; values are dicts</span> <span class="sd"> whose keys are package names and values are lists of either</span> <span class="sd"> dependencies for each package the symbols provided by each</span> <span class="sd"> package.</span> <span class="sd"> :param dependencies: A dict of dependencies found in the</span> <span class="sd"> metadata for this source.</span> <span class="sd"> :type dependencies: dict; see above.</span> <span class="sd"> :param provides: A dict of symbols provided by packages in</span> <span class="sd"> this repository.</span> <span class="sd"> :type provides: dict; see above.</span> <span class="sd"> """</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="s">'global'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="s">'global'</span><span class="p">]</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">dependencies</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="n">barch</span><span class="p">]</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">]</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span> <span class="k">for</span> <span class="n">pkgname</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span><span class="p">:</span> <span class="n">pset</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span> <span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">dependencies</span><span class="p">:</span> <span class="k">if</span> <span class="n">pkgname</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">dependencies</span><span class="p">[</span><span class="n">barch</span><span class="p">]:</span> <span class="n">dependencies</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">pset</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">dependencies</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]))</span> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">pset</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="s">'global'</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]</span> <span class="o">=</span> <span class="n">pset</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span> <span class="k">else</span><span class="p">:</span> <span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">dependencies</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]</span> <span class="o">=</span> <span class="n">dependencies</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">pkgname</span><span class="p">]</span> <span class="n">provided</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span> <span class="k">for</span> <span class="n">bprovided</span> <span class="ow">in</span> <span class="nb">list</span><span class="p">(</span><span class="n">provides</span><span class="o">.</span><span class="n">values</span><span class="p">()):</span> <span class="n">provided</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">bprovided</span><span class="p">))</span> <span class="k">for</span> <span class="n">prov</span> <span class="ow">in</span> <span class="n">provided</span><span class="p">:</span> <span class="n">prset</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span> <span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">provides</span><span class="p">:</span> <span class="k">if</span> <span class="n">prov</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">]:</span> <span class="k">continue</span> <span class="n">prset</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">]</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">prov</span><span class="p">,</span> <span class="p">())))</span> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">prset</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="s">'global'</span><span class="p">][</span><span class="n">prov</span><span class="p">]</span> <span class="o">=</span> <span class="n">prset</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span> <span class="k">else</span><span class="p">:</span> <span class="k">for</span> <span class="n">barch</span> <span class="ow">in</span> <span class="n">provides</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">][</span><span class="n">prov</span><span class="p">]</span> <span class="o">=</span> <span class="n">provides</span><span class="p">[</span><span class="n">barch</span><span class="p">]</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">prov</span><span class="p">,</span> <span class="p">())</span> <span class="bp">self</span><span class="o">.</span><span class="n">save_state</span><span class="p">()</span> </div> <div class="viewcode-block" id="Source.unknown_filter"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.unknown_filter">[docs]</a> <span class="k">def</span> <span class="nf">unknown_filter</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span> <span class="sd">""" A predicate that is used by :func:`filter_unknown` to</span> <span class="sd"> filter packages from the results of</span> <span class="sd"> :func:`Bcfg2.Server.Plugins.Packages.Collection.Collection.complete`</span> <span class="sd"> that should not be shown to the end user (i.e., that are not</span> <span class="sd"> truly unknown, but are rather packaging system artifacts). By</span> <span class="sd"> default, excludes any package whose name starts with "choice"</span> <span class="sd"> :param package: The name of a package that was unknown to the</span> <span class="sd"> backend</span> <span class="sd"> :type package: string</span> <span class="sd"> :returns: bool</span> <span class="sd"> """</span> <span class="k">return</span> <span class="n">package</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">"choice"</span><span class="p">)</span> </div> <div class="viewcode-block" id="Source.filter_unknown"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.filter_unknown">[docs]</a> <span class="k">def</span> <span class="nf">filter_unknown</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">unknown</span><span class="p">):</span> <span class="sd">""" After</span> <span class="sd"> :func:`Bcfg2.Server.Plugins.Packages.Collection.Collection.complete`,</span> <span class="sd"> filter out packages that appear in the list of unknown</span> <span class="sd"> packages but should not be presented to the user.</span> <span class="sd"> :attr:`unknown_filter` is called to assess whether or not a</span> <span class="sd"> package is expected to be unknown.</span> <span class="sd"> :param unknown: A set of unknown packages. The set should be</span> <span class="sd"> modified in place.</span> <span class="sd"> :type unknown: set of strings</span> <span class="sd"> """</span> <span class="n">unknown</span><span class="o">.</span><span class="n">difference_update</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">u</span> <span class="k">for</span> <span class="n">u</span> <span class="ow">in</span> <span class="n">unknown</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">unknown_filter</span><span class="p">(</span><span class="n">u</span><span class="p">)]))</span> </div> <div class="viewcode-block" id="Source.update"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.update">[docs]</a> <span class="k">def</span> <span class="nf">update</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="sd">""" Download metadata from the upstream repository and cache</span> <span class="sd"> it locally.</span> <span class="sd"> :raises: ValueError - If any URL in :attr:`urls` is malformed</span> <span class="sd"> :raises: OSError - If there is an error writing the local</span> <span class="sd"> cache</span> <span class="sd"> :raises: HTTPError - If there is an error fetching the remote</span> <span class="sd"> data</span> <span class="sd"> """</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">urls</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s">"Packages: Updating </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">url</span><span class="p">)</span> <span class="n">fname</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">escape_url</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="k">try</span><span class="p">:</span> <span class="nb">open</span><span class="p">(</span><span class="n">fname</span><span class="p">,</span> <span class="s">'w'</span><span class="p">)</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">fetch_url</span><span class="p">(</span><span class="n">url</span><span class="p">))</span> <span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Bad url string </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">url</span><span class="p">)</span> <span class="k">raise</span> <span class="k">except</span> <span class="ne">OSError</span><span class="p">:</span> <span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Could not write data from </span><span class="si">%s</span><span class="s"> to "</span> <span class="s">"local cache at </span><span class="si">%s</span><span class="s">: </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">fname</span><span class="p">,</span> <span class="n">err</span><span class="p">))</span> <span class="k">raise</span> <span class="k">except</span> <span class="n">HTTPError</span><span class="p">:</span> <span class="n">err</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span> <span class="bp">self</span><span class="o">.</span><span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">"Packages: Failed to fetch url </span><span class="si">%s</span><span class="s">. HTTP "</span> <span class="s">"response code=</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">err</span><span class="o">.</span><span class="n">code</span><span class="p">))</span> <span class="k">raise</span> </div> <div class="viewcode-block" id="Source.applies"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.applies">[docs]</a> <span class="k">def</span> <span class="nf">applies</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span> <span class="sd">""" Return true if this source applies to the given client,</span> <span class="sd"> i.e., the client is in all necessary groups and</span> <span class="sd"> :ref:`server-plugins-generators-packages-magic-groups`.</span> <span class="sd"> :param metadata: The client metadata to check to see if this</span> <span class="sd"> source applies</span> <span class="sd"> :type metadata: Bcfg2.Server.Plugins.Metadata.ClientMetadata</span> <span class="sd"> :returns: bool</span> <span class="sd"> """</span> <span class="c"># check base groups</span> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">magic_groups_match</span><span class="p">(</span><span class="n">metadata</span><span class="p">):</span> <span class="k">return</span> <span class="bp">False</span> <span class="c"># check Group/Client tags from sources.xml</span> <span class="k">for</span> <span class="n">condition</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">conditions</span><span class="p">:</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">(</span><span class="n">metadata</span><span class="p">):</span> <span class="k">return</span> <span class="bp">False</span> <span class="k">return</span> <span class="bp">True</span> </div> <div class="viewcode-block" id="Source.get_arches"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_arches">[docs]</a> <span class="k">def</span> <span class="nf">get_arches</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span> <span class="sd">""" Get a list of architectures that the given client has and</span> <span class="sd"> for which this source provides packages for. The return value</span> <span class="sd"> will always include ``global``.</span> <span class="sd"> :param metadata: The client metadata to get matching</span> <span class="sd"> architectures for</span> <span class="sd"> :type metadata: Bcfg2.Server.Plugins.Metadata.ClientMetadata</span> <span class="sd"> :returns: list of strings</span> <span class="sd"> """</span> <span class="k">return</span> <span class="p">[</span><span class="s">'global'</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">a</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span> <span class="k">if</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span><span class="p">]</span> </div> <div class="viewcode-block" id="Source.get_deps"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_deps">[docs]</a> <span class="k">def</span> <span class="nf">get_deps</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span> <span class="sd">""" Get a list of the dependencies of the given package.</span> <span class="sd"> :param package: The name of the symbol</span> <span class="sd"> :type package: string</span> <span class="sd"> :returns: list of strings</span> <span class="sd"> """</span> <span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_arches</span><span class="p">(</span><span class="n">metadata</span><span class="p">):</span> <span class="k">if</span> <span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="n">arch</span><span class="p">]:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">deps</span><span class="p">[</span><span class="n">arch</span><span class="p">][</span><span class="n">package</span><span class="p">]</span> <span class="k">return</span> <span class="p">[]</span> </div> <div class="viewcode-block" id="Source.get_provides"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_provides">[docs]</a> <span class="k">def</span> <span class="nf">get_provides</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span> <span class="sd">""" Get a list of all symbols provided by the given package.</span> <span class="sd"> :param package: The name of the package</span> <span class="sd"> :type package: string</span> <span class="sd"> :returns: list of strings</span> <span class="sd"> """</span> <span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_arches</span><span class="p">(</span><span class="n">metadata</span><span class="p">):</span> <span class="k">if</span> <span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">arch</span><span class="p">]:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">provides</span><span class="p">[</span><span class="n">arch</span><span class="p">][</span><span class="n">package</span><span class="p">]</span> <span class="k">return</span> <span class="p">[]</span> </div> <div class="viewcode-block" id="Source.is_package"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.is_package">[docs]</a> <span class="k">def</span> <span class="nf">is_package</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">package</span><span class="p">):</span> <span class="c"># pylint: disable=W0613</span> <span class="sd">""" Return True if a package is a package, False otherwise.</span> <span class="sd"> :param package: The name of the package</span> <span class="sd"> :type package: string</span> <span class="sd"> :returns: bool</span> <span class="sd"> """</span> <span class="k">return</span> <span class="p">(</span><span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">pkgnames</span> <span class="ow">and</span> <span class="n">package</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">blacklist</span> <span class="ow">and</span> <span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">whitelist</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">whitelist</span><span class="p">))</span> </div> <div class="viewcode-block" id="Source.get_group"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.get_group">[docs]</a> <span class="k">def</span> <span class="nf">get_group</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">,</span> <span class="n">group</span><span class="p">,</span> <span class="n">ptype</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span> <span class="c"># pylint: disable=W0613</span> <span class="sd">""" Get the list of packages of the given type in a package</span> <span class="sd"> group.</span> <span class="sd"> :param group: The name of the group to query</span> <span class="sd"> :type group: string</span> <span class="sd"> :param ptype: The type of packages to get, for backends that</span> <span class="sd"> support multiple package types in package groups</span> <span class="sd"> (e.g., "recommended," "optional," etc.)</span> <span class="sd"> :type ptype: string</span> <span class="sd"> :returns: list of strings - package names</span> <span class="sd"> """</span> <span class="k">return</span> <span class="p">[]</span> </div> <div class="viewcode-block" id="Source.magic_groups_match"><a class="viewcode-back" href="../../../../../development/packages.html#Bcfg2.Server.Plugins.Packages.Source.Source.magic_groups_match">[docs]</a> <span class="k">def</span> <span class="nf">magic_groups_match</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">metadata</span><span class="p">):</span> <span class="sd">""" Returns True if the client's</span> <span class="sd"> :ref:`server-plugins-generators-packages-magic-groups` match</span> <span class="sd"> the magic groups this source. Also returns True if magic</span> <span class="sd"> groups are off in the configuration and the client's</span> <span class="sd"> architecture matches (i.e., architecture groups are *always*</span> <span class="sd"> checked).</span> <span class="sd"> :returns: bool</span> <span class="sd"> """</span> <span class="n">found_arch</span> <span class="o">=</span> <span class="bp">False</span> <span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arches</span><span class="p">:</span> <span class="k">if</span> <span class="n">arch</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span><span class="p">:</span> <span class="n">found_arch</span> <span class="o">=</span> <span class="bp">True</span> <span class="k">break</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">found_arch</span><span class="p">:</span> <span class="k">return</span> <span class="bp">False</span> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">setup</span><span class="o">.</span><span class="n">cfp</span><span class="o">.</span><span class="n">getboolean</span><span class="p">(</span><span class="s">"packages"</span><span class="p">,</span> <span class="s">"magic_groups"</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="bp">False</span><span class="p">):</span> <span class="k">return</span> <span class="bp">True</span> <span class="k">else</span><span class="p">:</span> <span class="k">for</span> <span class="n">group</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">basegroups</span><span class="p">:</span> <span class="k">if</span> <span class="n">group</span> <span class="ow">in</span> <span class="n">metadata</span><span class="o">.</span><span class="n">groups</span><span class="p">:</span> <span class="k">return</span> <span class="bp">True</span> <span class="k">return</span> <span class="bp">False</span></div></div> </pre></div> </div> </div> </div> <div class="sphinxsidebar"> <div class="sphinxsidebarwrapper"> <div id="searchbox" style="display: none"> <h3>Quick search</h3> <form class="search" action="../../../../../search.html" method="get"> <input type="text" name="q" /> <input type="submit" value="Go" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> <p class="searchtip" style="font-size: 90%"> Enter search terms or a module, class or function name. </p> </div> <script type="text/javascript">$('#searchbox').show(0);</script> </div> </div> <div class="clearer"></div> </div> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="../../../../../genindex.html" title="General Index" >index</a></li> <li class="right" > <a href="../../../../../py-modindex.html" title="Python Module Index" >modules</a> |</li> <li><a href="../../../../../index.html">home</a> | </li> <!--<li><a href="../../../../../search.html">search</a> | </li>--> <li><a href="../../../../../help/index.html">help</a> | </li> <li><a href="../../../../../contents.html">documentation </a> »</li> <li><a href="../../../../index.html" >Module code</a> »</li> <li><a href="../../Plugins.html" >Bcfg2.Server.Plugins</a> »</li> <li><a href="../Packages.html" >Bcfg2.Server.Plugins.Packages</a> »</li> </ul> </div> <div class="footer"> © Copyright 2009-2013, Narayan Desai. Last updated on Mar 20, 2013. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3. </div> </body> </html>