<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html> <head> <title>The full Pygments API — Pygments</title> <meta http-equiv="content-type" content="text/html; charset=utf-8"> <style type="text/css"> body { background-color: #f2f2f2; margin: 0; padding: 0; font-family: 'Georgia', serif; color: #111; } #content { background-color: white; padding: 20px; margin: 20px auto 20px auto; max-width: 800px; border: 4px solid #ddd; } h1 { font-weight: normal; font-size: 40px; color: #09839A; } h2 { font-weight: normal; font-size: 30px; color: #C73F00; } h1.heading { margin: 0 0 30px 0; } h2.subheading { margin: -30px 0 0 45px; } h3 { margin-top: 30px; } table.docutils { border-collapse: collapse; border: 2px solid #aaa; margin: 0.5em 1.5em 0.5em 1.5em; } table.docutils td { padding: 2px; border: 1px solid #ddd; } p, li, dd, dt, blockquote { font-size: 15px; color: #333; } p { line-height: 150%; margin-bottom: 0; margin-top: 10px; } hr { border-top: 1px solid #ccc; border-bottom: 0; border-right: 0; border-left: 0; margin-bottom: 10px; margin-top: 20px; } dl { margin-left: 10px; } li, dt { margin-top: 5px; } dt { font-weight: bold; } th { text-align: left; } a { color: #990000; } a:hover { color: #c73f00; } pre { background-color: #f9f9f9; border-top: 1px solid #ccc; border-bottom: 1px solid #ccc; padding: 5px; font-size: 13px; font-family: Bitstream Vera Sans Mono,monospace; } tt { font-size: 13px; font-family: Bitstream Vera Sans Mono,monospace; color: black; padding: 1px 2px 1px 2px; background-color: #f0f0f0; } cite { /* abusing <cite>, it's generated by ReST for `x` */ font-size: 13px; font-family: Bitstream Vera Sans Mono,monospace; font-weight: bold; font-style: normal; } #backlink { float: right; font-size: 11px; color: #888; } div.toc { margin: 0 0 10px 0; } div.toc h2 { font-size: 20px; } .syntax .hll { background-color: #ffffcc } .syntax { background: #ffffff; } .syntax .c { color: #888888 } /* Comment */ .syntax .err { color: #a61717; background-color: #e3d2d2 } /* Error */ .syntax .k { color: #008800; font-weight: bold } /* Keyword */ .syntax .cm { color: #888888 } /* Comment.Multiline */ .syntax .cp { color: #cc0000; font-weight: bold } /* Comment.Preproc */ .syntax .c1 { color: #888888 } /* Comment.Single */ .syntax .cs { color: #cc0000; font-weight: bold; background-color: #fff0f0 } /* Comment.Special */ .syntax .gd { color: #000000; background-color: #ffdddd } /* Generic.Deleted */ .syntax .ge { font-style: italic } /* Generic.Emph */ .syntax .gr { color: #aa0000 } /* Generic.Error */ .syntax .gh { color: #303030 } /* Generic.Heading */ .syntax .gi { color: #000000; background-color: #ddffdd } /* Generic.Inserted */ .syntax .go { color: #888888 } /* Generic.Output */ .syntax .gp { color: #555555 } /* Generic.Prompt */ .syntax .gs { font-weight: bold } /* Generic.Strong */ .syntax .gu { color: #606060 } /* Generic.Subheading */ .syntax .gt { color: #aa0000 } /* Generic.Traceback */ .syntax .kc { color: #008800; font-weight: bold } /* Keyword.Constant */ .syntax .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */ .syntax .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */ .syntax .kp { color: #008800 } /* Keyword.Pseudo */ .syntax .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */ .syntax .kt { color: #888888; font-weight: bold } /* Keyword.Type */ .syntax .m { color: #0000DD; font-weight: bold } /* Literal.Number */ .syntax .s { color: #dd2200; background-color: #fff0f0 } /* Literal.String */ .syntax .na { color: #336699 } /* Name.Attribute */ .syntax .nb { color: #003388 } /* Name.Builtin */ .syntax .nc { color: #bb0066; font-weight: bold } /* Name.Class */ .syntax .no { color: #003366; font-weight: bold } /* Name.Constant */ .syntax .nd { color: #555555 } /* Name.Decorator */ .syntax .ne { color: #bb0066; font-weight: bold } /* Name.Exception */ .syntax .nf { color: #0066bb; font-weight: bold } /* Name.Function */ .syntax .nl { color: #336699; font-style: italic } /* Name.Label */ .syntax .nn { color: #bb0066; font-weight: bold } /* Name.Namespace */ .syntax .py { color: #336699; font-weight: bold } /* Name.Property */ .syntax .nt { color: #bb0066; font-weight: bold } /* Name.Tag */ .syntax .nv { color: #336699 } /* Name.Variable */ .syntax .ow { color: #008800 } /* Operator.Word */ .syntax .w { color: #bbbbbb } /* Text.Whitespace */ .syntax .mf { color: #0000DD; font-weight: bold } /* Literal.Number.Float */ .syntax .mh { color: #0000DD; font-weight: bold } /* Literal.Number.Hex */ .syntax .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */ .syntax .mo { color: #0000DD; font-weight: bold } /* Literal.Number.Oct */ .syntax .sb { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Backtick */ .syntax .sc { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Char */ .syntax .sd { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Doc */ .syntax .s2 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Double */ .syntax .se { color: #0044dd; background-color: #fff0f0 } /* Literal.String.Escape */ .syntax .sh { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Heredoc */ .syntax .si { color: #3333bb; background-color: #fff0f0 } /* Literal.String.Interpol */ .syntax .sx { color: #22bb22; background-color: #f0fff0 } /* Literal.String.Other */ .syntax .sr { color: #008800; background-color: #fff0ff } /* Literal.String.Regex */ .syntax .s1 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Single */ .syntax .ss { color: #aa6600; background-color: #fff0f0 } /* Literal.String.Symbol */ .syntax .bp { color: #003388 } /* Name.Builtin.Pseudo */ .syntax .vc { color: #336699 } /* Name.Variable.Class */ .syntax .vg { color: #dd7700 } /* Name.Variable.Global */ .syntax .vi { color: #3333bb } /* Name.Variable.Instance */ .syntax .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */ </style> </head> <body> <div id="content"> <h1 class="heading">Pygments</h1> <h2 class="subheading">The full Pygments API</h2> <a id="backlink" href="index.html">« Back To Index</a> <div class="toc"> <h2>Contents</h2> <ul class="contents"> <li><a href="#high-level-api">High-level API</a></li> <li><a href="#lexers">Lexers</a></li> <li><a href="#formatters">Formatters</a></li> <li><a href="#option-processing">Option processing</a></li> </ul> </div> <!-- -*- mode: rst -*- --> <p>This page describes the Pygments API.</p> <div class="section" id="high-level-api"> <h3>High-level API</h3> <p>Functions from the <cite>pygments</cite> module:</p> <dl class="docutils"> <dt>def <cite>lex(code, lexer):</cite></dt> <dd>Lex <cite>code</cite> with the <cite>lexer</cite> (must be a <cite>Lexer</cite> instance) and return an iterable of tokens. Currently, this only calls <cite>lexer.get_tokens()</cite>.</dd> <dt>def <cite>format(tokens, formatter, outfile=None):</cite></dt> <dd>Format a token stream (iterable of tokens) <cite>tokens</cite> with the <cite>formatter</cite> (must be a <cite>Formatter</cite> instance). The result is written to <cite>outfile</cite>, or if that is <tt class="docutils literal">None</tt>, returned as a string.</dd> <dt>def <cite>highlight(code, lexer, formatter, outfile=None):</cite></dt> <dd>This is the most high-level highlighting function. It combines <cite>lex</cite> and <cite>format</cite> in one function.</dd> </dl> <p>Functions from <cite>pygments.lexers</cite>:</p> <dl class="docutils"> <dt>def <cite>get_lexer_by_name(alias, **options):</cite></dt> <dd><p class="first">Return an instance of a <cite>Lexer</cite> subclass that has <cite>alias</cite> in its aliases list. The lexer is given the <cite>options</cite> at its instantiation.</p> <p class="last">Will raise <cite>pygments.util.ClassNotFound</cite> if no lexer with that alias is found.</p> </dd> <dt>def <cite>get_lexer_for_filename(fn, **options):</cite></dt> <dd><p class="first">Return a <cite>Lexer</cite> subclass instance that has a filename pattern matching <cite>fn</cite>. The lexer is given the <cite>options</cite> at its instantiation.</p> <p class="last">Will raise <cite>pygments.util.ClassNotFound</cite> if no lexer for that filename is found.</p> </dd> <dt>def <cite>get_lexer_for_mimetype(mime, **options):</cite></dt> <dd><p class="first">Return a <cite>Lexer</cite> subclass instance that has <cite>mime</cite> in its mimetype list. The lexer is given the <cite>options</cite> at its instantiation.</p> <p class="last">Will raise <cite>pygments.util.ClassNotFound</cite> if not lexer for that mimetype is found.</p> </dd> <dt>def <cite>guess_lexer(text, **options):</cite></dt> <dd><p class="first">Return a <cite>Lexer</cite> subclass instance that's guessed from the text in <cite>text</cite>. For that, the <cite>analyse_text()</cite> method of every known lexer class is called with the text as argument, and the lexer which returned the highest value will be instantiated and returned.</p> <p class="last"><cite>pygments.util.ClassNotFound</cite> is raised if no lexer thinks it can handle the content.</p> </dd> <dt>def <cite>guess_lexer_for_filename(filename, text, **options):</cite></dt> <dd><p class="first">As <cite>guess_lexer()</cite>, but only lexers which have a pattern in <cite>filenames</cite> or <cite>alias_filenames</cite> that matches <cite>filename</cite> are taken into consideration.</p> <p class="last"><cite>pygments.util.ClassNotFound</cite> is raised if no lexer thinks it can handle the content.</p> </dd> <dt>def <cite>get_all_lexers():</cite></dt> <dd><p class="first">Return an iterable over all registered lexers, yielding tuples in the format:</p> <pre class="literal-block"> (longname, tuple of aliases, tuple of filename patterns, tuple of mimetypes) </pre> <p class="last"><em>New in Pygments 0.6.</em></p> </dd> </dl> <p>Functions from <cite>pygments.formatters</cite>:</p> <dl class="docutils"> <dt>def <cite>get_formatter_by_name(alias, **options):</cite></dt> <dd><p class="first">Return an instance of a <cite>Formatter</cite> subclass that has <cite>alias</cite> in its aliases list. The formatter is given the <cite>options</cite> at its instantiation.</p> <p class="last">Will raise <cite>pygments.util.ClassNotFound</cite> if no formatter with that alias is found.</p> </dd> <dt>def <cite>get_formatter_for_filename(fn, **options):</cite></dt> <dd><p class="first">Return a <cite>Formatter</cite> subclass instance that has a filename pattern matching <cite>fn</cite>. The formatter is given the <cite>options</cite> at its instantiation.</p> <p class="last">Will raise <cite>pygments.util.ClassNotFound</cite> if no formatter for that filename is found.</p> </dd> </dl> <p>Functions from <cite>pygments.styles</cite>:</p> <dl class="docutils"> <dt>def <cite>get_style_by_name(name):</cite></dt> <dd><p class="first">Return a style class by its short name. The names of the builtin styles are listed in <cite>pygments.styles.STYLE_MAP</cite>.</p> <p class="last">Will raise <cite>pygments.util.ClassNotFound</cite> if no style of that name is found.</p> </dd> <dt>def <cite>get_all_styles():</cite></dt> <dd><p class="first">Return an iterable over all registered styles, yielding their names.</p> <p class="last"><em>New in Pygments 0.6.</em></p> </dd> </dl> </div> <div class="section" id="lexers"> <h3>Lexers</h3> <p>A lexer (derived from <cite>pygments.lexer.Lexer</cite>) has the following functions:</p> <dl class="docutils"> <dt>def <cite>__init__(self, **options):</cite></dt> <dd><p class="first">The constructor. Takes a **keywords dictionary of options. Every subclass must first process its own options and then call the <cite>Lexer</cite> constructor, since it processes the <cite>stripnl</cite>, <cite>stripall</cite> and <cite>tabsize</cite> options.</p> <p>An example looks like this:</p> <div class="syntax"><pre><span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">options</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">compress</span> <span class="o">=</span> <span class="n">options</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'compress'</span><span class="p">,</span> <span class="s">''</span><span class="p">)</span> <span class="n">Lexer</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">options</span><span class="p">)</span> </pre></div> <p class="last">As these options must all be specifiable as strings (due to the command line usage), there are various utility functions available to help with that, see <a class="reference internal" href="#option-processing">Option processing</a>.</p> </dd> <dt>def <cite>get_tokens(self, text):</cite></dt> <dd><p class="first">This method is the basic interface of a lexer. It is called by the <cite>highlight()</cite> function. It must process the text and return an iterable of <tt class="docutils literal">(tokentype, value)</tt> pairs from <cite>text</cite>.</p> <p class="last">Normally, you don't need to override this method. The default implementation processes the <cite>stripnl</cite>, <cite>stripall</cite> and <cite>tabsize</cite> options and then yields all tokens from <cite>get_tokens_unprocessed()</cite>, with the <tt class="docutils literal">index</tt> dropped.</p> </dd> <dt>def <cite>get_tokens_unprocessed(self, text):</cite></dt> <dd><p class="first">This method should process the text and return an iterable of <tt class="docutils literal">(index, tokentype, value)</tt> tuples where <tt class="docutils literal">index</tt> is the starting position of the token within the input text.</p> <p class="last">This method must be overridden by subclasses.</p> </dd> <dt>def <cite>analyse_text(text):</cite></dt> <dd>A static method which is called for lexer guessing. It should analyse the text and return a float in the range from <tt class="docutils literal">0.0</tt> to <tt class="docutils literal">1.0</tt>. If it returns <tt class="docutils literal">0.0</tt>, the lexer will not be selected as the most probable one, if it returns <tt class="docutils literal">1.0</tt>, it will be selected immediately.</dd> </dl> <p>For a list of known tokens have a look at the <a class="reference external" href="./tokens.html">Tokens</a> page.</p> <p>A lexer also can have the following attributes (in fact, they are mandatory except <cite>alias_filenames</cite>) that are used by the builtin lookup mechanism.</p> <dl class="docutils"> <dt><cite>name</cite></dt> <dd>Full name for the lexer, in human-readable form.</dd> <dt><cite>aliases</cite></dt> <dd>A list of short, unique identifiers that can be used to lookup the lexer from a list, e.g. using <cite>get_lexer_by_name()</cite>.</dd> <dt><cite>filenames</cite></dt> <dd>A list of <cite>fnmatch</cite> patterns that match filenames which contain content for this lexer. The patterns in this list should be unique among all lexers.</dd> <dt><cite>alias_filenames</cite></dt> <dd>A list of <cite>fnmatch</cite> patterns that match filenames which may or may not contain content for this lexer. This list is used by the <cite>guess_lexer_for_filename()</cite> function, to determine which lexers are then included in guessing the correct one. That means that e.g. every lexer for HTML and a template language should include <tt class="docutils literal"><span class="pre">\*.html</span></tt> in this list.</dd> <dt><cite>mimetypes</cite></dt> <dd>A list of MIME types for content that can be lexed with this lexer.</dd> </dl> </div> <div class="section" id="formatters"> <h3>Formatters</h3> <p>A formatter (derived from <cite>pygments.formatter.Formatter</cite>) has the following functions:</p> <dl class="docutils"> <dt>def <cite>__init__(self, **options):</cite></dt> <dd><p class="first">As with lexers, this constructor processes options and then must call the base class <cite>__init__</cite>.</p> <p class="last">The <cite>Formatter</cite> class recognizes the options <cite>style</cite>, <cite>full</cite> and <cite>title</cite>. It is up to the formatter class whether it uses them.</p> </dd> <dt>def <cite>get_style_defs(self, arg=''):</cite></dt> <dd><p class="first">This method must return statements or declarations suitable to define the current style for subsequent highlighted text (e.g. CSS classes in the <cite>HTMLFormatter</cite>).</p> <p>The optional argument <cite>arg</cite> can be used to modify the generation and is formatter dependent (it is standardized because it can be given on the command line).</p> <p class="last">This method is called by the <tt class="docutils literal"><span class="pre">-S</span></tt> <a class="reference external" href="./cmdline.html">command-line option</a>, the <cite>arg</cite> is then given by the <tt class="docutils literal"><span class="pre">-a</span></tt> option.</p> </dd> <dt>def <cite>format(self, tokensource, outfile):</cite></dt> <dd><p class="first">This method must format the tokens from the <cite>tokensource</cite> iterable and write the formatted version to the file object <cite>outfile</cite>.</p> <p class="last">Formatter options can control how exactly the tokens are converted.</p> </dd> </dl> <p>A formatter must have the following attributes that are used by the builtin lookup mechanism. (<em>New in Pygments 0.7.</em>)</p> <dl class="docutils"> <dt><cite>name</cite></dt> <dd>Full name for the formatter, in human-readable form.</dd> <dt><cite>aliases</cite></dt> <dd>A list of short, unique identifiers that can be used to lookup the formatter from a list, e.g. using <cite>get_formatter_by_name()</cite>.</dd> <dt><cite>filenames</cite></dt> <dd>A list of <cite>fnmatch</cite> patterns that match filenames for which this formatter can produce output. The patterns in this list should be unique among all formatters.</dd> </dl> </div> <div class="section" id="option-processing"> <h3>Option processing</h3> <p>The <cite>pygments.util</cite> module has some utility functions usable for option processing:</p> <dl class="docutils"> <dt>class <cite>OptionError</cite></dt> <dd>This exception will be raised by all option processing functions if the type or value of the argument is not correct.</dd> <dt>def <cite>get_bool_opt(options, optname, default=None):</cite></dt> <dd><p class="first">Interpret the key <cite>optname</cite> from the dictionary <cite>options</cite> as a boolean and return it. Return <cite>default</cite> if <cite>optname</cite> is not in <cite>options</cite>.</p> <p class="last">The valid string values for <tt class="docutils literal">True</tt> are <tt class="docutils literal">1</tt>, <tt class="docutils literal">yes</tt>, <tt class="docutils literal">true</tt> and <tt class="docutils literal">on</tt>, the ones for <tt class="docutils literal">False</tt> are <tt class="docutils literal">0</tt>, <tt class="docutils literal">no</tt>, <tt class="docutils literal">false</tt> and <tt class="docutils literal">off</tt> (matched case-insensitively).</p> </dd> <dt>def <cite>get_int_opt(options, optname, default=None):</cite></dt> <dd>As <cite>get_bool_opt</cite>, but interpret the value as an integer.</dd> <dt>def <cite>get_list_opt(options, optname, default=None):</cite></dt> <dd>If the key <cite>optname</cite> from the dictionary <cite>options</cite> is a string, split it at whitespace and return it. If it is already a list or a tuple, it is returned as a list.</dd> <dt>def <cite>get_choice_opt(options, optname, allowed, default=None):</cite></dt> <dd>If the key <cite>optname</cite> from the dictionary is not in the sequence <cite>allowed</cite>, raise an error, otherwise return it. <em>New in Pygments 0.8.</em></dd> </dl> </div> </div> </body> <!-- generated on: 2010-03-01 21:18:22.940790 file id: api --> </html>