<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Bazaar Windows EC2 Server — Bazaar 2.7.0 documentation</title> <link rel="stylesheet" href="_static/classic.css" type="text/css" /> <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: './', VERSION: '2.7.0', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="_static/jquery.js"></script> <script type="text/javascript" src="_static/underscore.js"></script> <script type="text/javascript" src="_static/doctools.js"></script> <link rel="shortcut icon" href="_static/bzr.ico"/> <link rel="search" title="Search" href="search.html" /> <link rel="top" title="Bazaar 2.7.0 documentation" href="index.html" /> <link rel="next" title="Bazaar Architectural Overview" href="overview.html" /> <link rel="prev" title="Managing the Bazaar PPA" href="ppa.html" /> <link rel="stylesheet" href="_static/bzr-doc.css" type="text/css" /> </head> <body role="document"> <div class="related" role="navigation" aria-label="related navigation"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="overview.html" title="Bazaar Architectural Overview" accesskey="N">next</a></li> <li class="right" > <a href="ppa.html" title="Managing the Bazaar PPA" accesskey="P">previous</a> |</li> <li><a href="http://bazaar.canonical.com/"> <img src="_static/bzr icon 16.png" /> Home</a> | </li> <a href="http://doc.bazaar.canonical.com/en/">Documentation</a> | </li> <li class="nav-item nav-item-0"><a href="index.html">Developer Document Catalog (2.7.0)</a> »</li> </ul> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body" role="main"> <div class="section" id="bazaar-windows-ec2-server"> <h1>Bazaar Windows EC2 Server<a class="headerlink" href="#bazaar-windows-ec2-server" title="Permalink to this headline">¶</a></h1> <p>We have an Amazon EC2 virtual machine called <a class="reference external" href="http://en.wikipedia.org/wiki/Desolation_Island">Desolation</a> for building Windows packages and general testing on Windows. As of 2009-02-19, this is just experimental and this is a draft specification, but we aim to use it for the production Windows installer build of 1.13 in March.</p> <p>See also:</p> <ul class="simple"> <li><a class="reference external" href="index.html">Bazaar Developer Documentation Catalog</a>.</li> </ul> <div class="section" id="goals"> <h2>Goals<a class="headerlink" href="#goals" title="Permalink to this headline">¶</a></h2> <ul class="simple"> <li>The instance is only running (and incurring charges) when it’s needed for testing or packaging.</li> <li>It can be started or stopped by anyone on the team using a straightforward script.</li> <li>Multiple people can get into the same instance at the same time, e.g. if one person needs to pass work on to some one else.</li> <li>We keep snapshot of the OS and tool chain so that we can roll back if we need to.</li> <li>bzr branches and similar information are kept on stable storage that survives rollbacks of the OS state, and that can be backed up.</li> </ul> <p>Later on we may try automated Windows testing in a similar setup.</p> </div> <div class="section" id="approach"> <h2>Approach<a class="headerlink" href="#approach" title="Permalink to this headline">¶</a></h2> <p>The working disk and the AMI images are stored in one person’s account for billing purposes.</p> <p>Ideally we want to give other people access to run this machine without giving full access to the account. I’m not sure if that’s feasible. If it’s not, we might need to allow people to launch the image within their own account; this may be problematic if the shared volume is already in use by someone else.</p> <p>I don’t think it’s possible to have an EBS that’s shared across accounts, and they can’t be attached to multiple running instances. So for now it’s probably best to just ignore the concept and store the working data on the instance’s local storage, and to copy things up e.g. to Launchpad as required.</p> <p>On this machine, <code class="docutils literal"><span class="pre">C:</span></code> should be used only for the Windows system files, <code class="docutils literal"><span class="pre">D:</span></code> for installed programs and working directories, and other drive letters can be used later for mounting EBS storage if desired.</p> <p>Through <code class="docutils literal"><span class="pre">ec2-modify-image-attribute</span></code> we can allow nominated users to access an existing image. We need to have their AWS opaque ID.</p> <p>Through <code class="docutils literal"><span class="pre">ec2-bundle-image</span></code> we can make a new snapshot at any point, which will be stored into the current user’s S3 account.</p> <p>We’ll (probably) have one shared account for running builds which is also an administrator for ease of installing software.</p> <p>You do need to have an RSA keypair to get the initial password for a Windows machine, even though you can’t use it to log in later. <code class="docutils literal"><span class="pre">ec2-get-password</span></code> takes the full path to the private key to obtain the password from Amazon, and <code class="docutils literal"><span class="pre">ec2-add-keypair</span></code> creates a named keypair at Amazon and returns the private path. One keypair is all that is needed. This is distinct from the account identifier - likely due to the different toolchains in use (the keypairs are used for unix SSH keys, and I (Robert) suspect a rather unix friendly core at Amazon). Once a custom image is made with a saved password, you can skip using <code class="docutils literal"><span class="pre">ec2-get-password</span></code> (which is only needed for Windows anyway).</p> <p>It would be nice if rdesktop could use private key authentication but apparently not.</p> <p>Should check how the Launchpad ec2test scripts work.</p> </div> <div class="section" id="procedures"> <h2>Procedures<a class="headerlink" href="#procedures" title="Permalink to this headline">¶</a></h2> <div class="section" id="preparation"> <h3>Preparation<a class="headerlink" href="#preparation" title="Permalink to this headline">¶</a></h3> <ul> <li><p class="first">Be in the bzr core team. If you are interested in helping with Windows packaging, testing or development just ask.</p> </li> <li><p class="first">Install the <a class="reference external" href="http://developer.amazonwebservices.com/connect/entry.jspa?externalID=368&categoryID=88">Amazon EC2 API tools</a> (needs-packaging <a class="reference external" href="https://bugs.launchpad.net/ubuntu/+bug/330930">bug 330930</a>)</p> </li> <li><p class="first">Create an Amazon Web Services account, sign up for S3 and EC2, and do the various steps to create authentication devices.</p> </li> <li><p class="first">Create a private key and certificate for yourself. Check these environment variables are set and exported, e.g. by setting them in the file <code class="docutils literal"><span class="pre">~/.aws</span></code>. Make sure the files are private.:</p> <div class="highlight-default"><div class="highlight"><pre><span></span>export EC2_PRIVATE_KEY=~/.ec2/pk-XXXXXX.pem export EC2_CERT=~/.ec2/cert-XXXXXX.pem export EC2_HOME=~/build/ec2-api-tools-1.3-30349 export AWS_SECRET_ACCESS_KEY=XXXXXXXXX export AWS_ACCESS_KEY_ID=XXXXXXXXXXX export EC2_KEYPAIR_NAME=XXXXXXXXX export PATH=$PATH:$EC2_HOME/bin export JAVA_HOME=/usr/lib/jvm/java-6-openjdk ssh-add ~/.ec2/id_rsa </pre></div> </div> <p>You can now ‘. ~/.aws’ to get the ec2 commands available.</p> </li> <li><p class="first">(Unix images only) run ec2-add-keypair SOMENAME, e.g. ‘bzr’. Put the result (minus the first line) somewhere like ~/.ec2/id_rsa and chmod go-rw.</p> </li> <li><p class="first">A useful Unix image is <a class="reference external" href="http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1762&categoryID=101">ami-bdfe19d4</a>, Eric Hammonds 64-bit Ubuntu image.</p> </li> <li><p class="first">Install the rdesktop client, to actually access the machine.</p> </li> <li><p class="first">Possibly read some of the <a class="reference external" href="http://aws.amazon.com/">EC2 documentation</a> for background.</p> </li> </ul> <ul> <li><p class="first">Create a security group for your that allows rdesktop access and icmp with:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">ec2</span><span class="o">-</span><span class="n">add</span><span class="o">-</span><span class="n">group</span> <span class="n">desolation</span><span class="o">-</span><span class="n">group</span> <span class="o">-</span><span class="n">d</span> <span class="s1">'bzr win32 build machine'</span> <span class="n">ec2</span><span class="o">-</span><span class="n">authorize</span> <span class="n">desolation</span><span class="o">-</span><span class="n">group</span> <span class="o">-</span><span class="n">p</span> <span class="mi">3389</span> <span class="o">-</span><span class="n">s</span> <span class="mf">1.2</span><span class="o">.</span><span class="mf">3.4</span><span class="o">/</span><span class="mi">32</span> <span class="n">ec2</span><span class="o">-</span><span class="n">authorize</span> <span class="n">desolation</span><span class="o">-</span><span class="n">group</span> <span class="o">-</span><span class="n">t</span> <span class="o">-</span><span class="mi">1</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span> <span class="o">-</span><span class="n">P</span> <span class="n">icmp</span> </pre></div> </div> <p>Add your public IP there. You can repeat that command to allow others in.</p> </li> </ul> </div> <div class="section" id="to-start-up-an-instance"> <h3>To start up an instance<a class="headerlink" href="#to-start-up-an-instance" title="Permalink to this headline">¶</a></h3> <ol class="arabic simple"> <li>Get the right AMI image ID from another developer.</li> </ol> <ol class="arabic"> <li><p class="first">Start the instance:</p> <div class="highlight-default"><div class="highlight"><pre><span></span>ec2-run-instances $image_id -g desolation-group </pre></div> </div> <p>This will print out some information including the image id, something like <code class="docutils literal"><span class="pre">i-31a74258</span></code>.</p> </li> </ol> <ol class="arabic"> <li><p class="first">Actually starting the machine will take a few minutes. Once it’s in the <em>running</em> state, get the machine’s public IP with</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">ec2</span><span class="o">-</span><span class="n">describe</span><span class="o">-</span><span class="n">instances</span> </pre></div> </div> </li> </ol> <ol class="arabic"> <li><p class="first">and then connect</p> <div class="highlight-default"><div class="highlight"><pre><span></span>rdesktop -g 1200x850 -u Administrator $machine_ip </pre></div> </div> </li> </ol> <p>Don’t forget to shut it down when you’re done, and check with <code class="docutils literal"><span class="pre">ec2-describe-instances</span></code> that it did terminate.</p> </div> <div class="section" id="to-save-a-system-snapshot-as-an-image"> <h3>To save a system snapshot as an image<a class="headerlink" href="#to-save-a-system-snapshot-as-an-image" title="Permalink to this headline">¶</a></h3> <ol class="arabic"> <li><p class="first">Bundle the current state. <em>Doing this will reboot the machine.</em> You need to choose a unique s3 bucket name, typically based on a domain or email address, which can contain any number of images. You also need a name unique within the bucket for this image, like <code class="docutils literal"><span class="pre">desolation-vs2008-20090219</span></code>. And finally it needs your AWS S3 access key and secret key, which should be set in <code class="docutils literal"><span class="pre">~/.aws</span></code>:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">ec2</span><span class="o">-</span><span class="n">bundle</span><span class="o">-</span><span class="n">instance</span> <span class="o">-</span><span class="n">b</span> <span class="n">ec2</span><span class="o">.</span><span class="n">sourcefrog</span><span class="o">.</span><span class="n">net</span> \ <span class="o">-</span><span class="n">p</span> <span class="n">desolation</span><span class="o">-</span><span class="n">vs2008</span><span class="o">-</span><span class="mi">2009021</span> \ <span class="o">-</span><span class="n">o</span> <span class="s2">"$AWS_ACCESS_KEY_ID"</span> \ <span class="o">-</span><span class="n">w</span> <span class="s2">"$AWS_SECRET_ACCESS_KEY"</span> </pre></div> </div> </li> </ol> <ol class="arabic"> <li><p class="first">This will take several minutes: You can check progress with</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">ec2</span><span class="o">-</span><span class="n">describe</span><span class="o">-</span><span class="n">bundle</span><span class="o">-</span><span class="n">tasks</span> </pre></div> </div> </li> </ol> <ol class="arabic"> <li><p class="first">Register the files as an image, e.g.:</p> <div class="highlight-default"><div class="highlight"><pre><span></span> <span class="n">ec2</span><span class="o">-</span><span class="n">register</span> <span class="n">ec2</span><span class="o">.</span><span class="n">sourcefrog</span><span class="o">.</span><span class="n">net</span><span class="o">/</span><span class="n">desolation</span><span class="o">-</span><span class="n">vs2008</span><span class="o">-</span><span class="mi">2009021</span> <span class="n">This</span> <span class="n">will</span> <span class="n">give</span> <span class="n">you</span> <span class="n">an</span> <span class="n">AMI</span> <span class="nb">id</span> <span class="k">for</span> <span class="n">the</span> <span class="n">image</span><span class="o">.</span> </pre></div> </div> </li> </ol> <ol class="arabic"> <li><p class="first">Give access to other team members identified by their Amazon account id:</p> <div class="highlight-default"><div class="highlight"><pre><span></span>ec2-modify-image-attributes $ami_id -l -a 123412341234 </pre></div> </div> </li> </ol> </div> <div class="section" id="management-console-useful"> <h3>Management console (useful!)<a class="headerlink" href="#management-console-useful" title="Permalink to this headline">¶</a></h3> <p><a class="reference external" href="https://console.aws.amazon.com/ec2/home">https://console.aws.amazon.com/ec2/home</a></p> </div> </div> </div> </div> </div> </div> <div class="sphinxsidebar" role="navigation" aria-label="main navigation"> <div class="sphinxsidebarwrapper"> <h3><a href="index.html">Table Of Contents</a></h3> <ul> <li><a class="reference internal" href="#">Bazaar Windows EC2 Server</a><ul> <li><a class="reference internal" href="#goals">Goals</a></li> <li><a class="reference internal" href="#approach">Approach</a></li> <li><a class="reference internal" href="#procedures">Procedures</a><ul> <li><a class="reference internal" href="#preparation">Preparation</a></li> <li><a class="reference internal" href="#to-start-up-an-instance">To start up an instance</a></li> <li><a class="reference internal" href="#to-save-a-system-snapshot-as-an-image">To save a system snapshot as an image</a></li> <li><a class="reference internal" href="#management-console-useful">Management console (useful!)</a></li> </ul> </li> </ul> </li> </ul> <h4>Previous topic</h4> <p class="topless"><a href="ppa.html" title="previous chapter">Managing the Bazaar PPA</a></p> <h4>Next topic</h4> <p class="topless"><a href="overview.html" title="next chapter">Bazaar Architectural Overview</a></p> <div role="note" aria-label="source link"> <h3>This Page</h3> <ul class="this-page-menu"> <li><a href="_sources/ec2.txt" rel="nofollow">Show Source</a></li> </ul> </div> <div id="searchbox" style="display: none" role="search"> <h3>Quick search</h3> <form class="search" action="search.html" method="get"> <div><input type="text" name="q" /></div> <div><input type="submit" value="Go" /></div> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> </div> <script type="text/javascript">$('#searchbox').show(0);</script> </div> </div> <div class="clearer"></div> </div> <div class="related" role="navigation" aria-label="related navigation"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="overview.html" title="Bazaar Architectural Overview" >next</a></li> <li class="right" > <a href="ppa.html" title="Managing the Bazaar PPA" >previous</a> |</li> <li><a href="http://bazaar.canonical.com/"> <img src="_static/bzr icon 16.png" /> Home</a> | </li> <a href="http://doc.bazaar.canonical.com/en/">Documentation</a> | </li> <li class="nav-item nav-item-0"><a href="index.html">Developer Document Catalog (2.7.0)</a> »</li> </ul> </div> <div class="footer" role="contentinfo"> © Copyright 2009-2011 Canonical Ltd. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.9. </div> </body> </html>