{"id":891,"date":"2012-10-24T12:20:14","date_gmt":"2012-10-24T10:20:14","guid":{"rendered":"http:\/\/oldblogs.uct.ac.za\/blog\/big-bytes\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick"},"modified":"2015-08-14T12:19:32","modified_gmt":"2015-08-14T10:19:32","slug":"poking-the-amazon-coud-with-a-stick","status":"publish","type":"post","link":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/","title":{"rendered":"Poking the Amazon Coud with a stick"},"content":{"rendered":"
We thought it would be interesting to mint ourselves some Amazon Cloud servers and put them to the test. \u00a0The results were quite interesting when compared to our home grown cluster.<\/div>\r\n
First order of business was to create 3 t1.micro instances (because they're free) and install OpenMPI. \u00a0This was a bit of a mission as the one we installed doesn't completely replace the default version provided by AMI which needed some environment changes. \u00a0We also set up hostbased authentication which required changing the default AMI authentication procedure and then dealing with the Amazon private DNS.<\/div>\r\n
We then used one of the C programs we'd concocted to test CPU speed and memory. \u00a0The program creates and manipulates a large array in memory and spreads the load over the cores provided by OpenMPI.<\/div>\r\n
As the t1.micro AMIs simulate a single core we emulated this on our cluster by spreading the load over 3 nodes allowing only 1 core per node. \u00a0We used the 200 and 400 series, allowing for the fact that our 400 series are fairly heterogeneous when it comes to CPU speed. \u00a0Hence we distinguished between the slowest and fastest nodes available in this series. \u00a0In the graph below a higher Y axis placement is worse (slower).<\/div>\r\n
\"AWS<\/div>\r\n
The t1.micro instance was faster than our 200 series (3.6GHz but relatively small CPU cache) and 400 series (1.6GHz 12MB smart cache) servers, but as we increased the number of computations the time to completion increased considerably. \u00a0Our fastest 400 series servers (2.8GHz) ultimately proved the fastest.<\/div>\r\n
According to the AWS site<\/a>\u00a0one EC2 Compute Unit provides the equivalent CPU capacity an early-2006 1.7 GHz Xeon processor. \u00a0The t1.micro instance provide a small amount of consistent CPU resources and allow you to increase CPU capacity in short bursts when additional cycles are available.<\/div>\r\n
This explains why for short jobs the t1.micro instances outperformed our cluster but as complexity increased the AWS scheduler decreases the relative speed of t1.micro cores. \u00a0While we'd never consider the micro instances for real HPC it does bode well for the higher end AWS products. \u00a0Issues still to be addressed though are cost per cpu cycles over time and data transfer to and from the cloud.<\/div>","protected":false},"excerpt":{"rendered":"
We thought it would be interesting to mint ourselves some Amazon Cloud servers and put them to the test.  The results were quite interesting when compared to our home grown cluster.<\/div>\n
First order of business was to create 3 t1.micro instances (because they’re free) and install OpenMPI.  This was a bit of a mission as the one we installed doesn’t completely replace the default version provided by AMI which needed some environment changes.  We also set up hostbased authentication which required changing the default AMI authentication procedure and then dealing with the Amazon private DNS.<\/div>\n
We then used one of the C programs we’d concocted to test CPU speed and memory.  The program creates and manipulates a large array in memory and spreads the load over the cores provided by OpenMPI.<\/div>\n
As the t1.micro AMIs simulate a single core we emulated this on our cluster by spreading the load over 3 nodes allowing only 1 core per node.  We used the 200 and 400 series, allowing for the fact that our 400 series are fairly heterogeneous when it comes to CPU speed.  Hence we distinguished between the slowest and fastest nodes available in this series.  In the graph below a higher Y axis placement is worse (slower).<\/div>\n
\"AWS <\/div>\n
The t1.micro instance was faster than our 200 series (3.6GHz but relatively small CPU cache) and 400 series (1.6GHz 12MB smart cache) servers, but as we increased the number of computations the time to completion increased considerably.  Our fastest 400 series servers (2.8GHz) ultimately proved the fastest.<\/div>\n
According to the AWS site<\/a> one EC2 Compute Unit provides the equivalent CPU capacity an early-2006 1.7 GHz Xeon processor.  The t1.micro instance provide a small amount of consistent CPU resources and allow you to increase CPU capacity in short bursts when additional cycles are available. <\/div>\n
This explains why for short jobs the t1.micro instances outperformed our cluster but as complexity increased the AWS scheduler decreases the relative speed of t1.micro cores.  While we’d never consider the micro instances for real HPC it does bode well for the higher end AWS products.  Issues still to be addressed though are cost per cpu cycles over time and data transfer to and from the cloud.<\/div>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"yoast_head":"\nPoking the Amazon Coud with a stick - UCT HPC<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Poking the Amazon Coud with a stick - UCT HPC\" \/>\n<meta property=\"og:description\" content=\"We thought it would be interesting to mint ourselves some Amazon Cloud servers and put them to the test.  The results were quite interesting when compared to our home grown cluster.First order of business was to create 3 t1.micro instances (because they're free) and install OpenMPI.  This was a bit of a mission as the one we installed doesn't completely replace the default version provided by AMI which needed some environment changes.  We also set up hostbased authentication which required changing the default AMI authentication procedure and then dealing with the Amazon private DNS.We then used one of the C programs we'd concocted to test CPU speed and memory.  The program creates and manipulates a large array in memory and spreads the load over the cores provided by OpenMPI.As the t1.micro AMIs simulate a single core we emulated this on our cluster by spreading the load over 3 nodes allowing only 1 core per node.  We used the 200 and 400 series, allowing for the fact that our 400 series are fairly heterogeneous when it comes to CPU speed.  Hence we distinguished between the slowest and fastest nodes available in this series.  In the graph below a higher Y axis placement is worse (slower). The t1.micro instance was faster than our 200 series (3.6GHz but relatively small CPU cache) and 400 series (1.6GHz 12MB smart cache) servers, but as we increased the number of computations the time to completion increased considerably.  Our fastest 400 series servers (2.8GHz) ultimately proved the fastest.According to the AWS site one EC2 Compute Unit provides the equivalent CPU capacity an early-2006 1.7 GHz Xeon processor.  The t1.micro instance provide a small amount of consistent CPU resources and allow you to increase CPU capacity in short bursts when additional cycles are available. This explains why for short jobs the t1.micro instances outperformed our cluster but as complexity increased the AWS scheduler decreases the relative speed of t1.micro cores.  While we'd never consider the micro instances for real HPC it does bode well for the higher end AWS products.  Issues still to be addressed though are cost per cpu cycles over time and data transfer to and from the cloud.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/\" \/>\n<meta property=\"og:site_name\" content=\"UCT HPC\" \/>\n<meta property=\"article:published_time\" content=\"2012-10-24T10:20:14+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2015-08-14T10:19:32+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/blogs.uct.ac.za\/gallery\/1253\/awscomp.jpg\" \/>\n<meta name=\"author\" content=\"Andrew Lewis\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Andrew Lewis\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/\"},\"author\":{\"name\":\"Andrew Lewis\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\"},\"headline\":\"Poking the Amazon Coud with a stick\",\"datePublished\":\"2012-10-24T10:20:14+00:00\",\"dateModified\":\"2015-08-14T10:19:32+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/\"},\"wordCount\":370,\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"articleSection\":[\"application\"],\"inLanguage\":\"en-ZA\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/\",\"name\":\"Poking the Amazon Coud with a stick - UCT HPC\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\"},\"datePublished\":\"2012-10-24T10:20:14+00:00\",\"dateModified\":\"2015-08-14T10:19:32+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/#breadcrumb\"},\"inLanguage\":\"en-ZA\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucthpc.uct.ac.za\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Poking the Amazon Coud with a stick\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"name\":\"UCT HPC\",\"description\":\"University of Cape Town High Performance Computing\",\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-ZA\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\",\"name\":\"University of Cape Town High Performance Computing\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"contentUrl\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"width\":450,\"height\":423,\"caption\":\"University of Cape Town High Performance Computing\"},\"image\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\",\"name\":\"Andrew Lewis\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"caption\":\"Andrew Lewis\"},\"sameAs\":[\"http:\/\/blogs.uct.ac.za\/blog\/big-bytes\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Poking the Amazon Coud with a stick - UCT HPC","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/","og_locale":"en_US","og_type":"article","og_title":"Poking the Amazon Coud with a stick - UCT HPC","og_description":"We thought it would be interesting to mint ourselves some Amazon Cloud servers and put them to the test.  The results were quite interesting when compared to our home grown cluster.First order of business was to create 3 t1.micro instances (because they're free) and install OpenMPI.  This was a bit of a mission as the one we installed doesn't completely replace the default version provided by AMI which needed some environment changes.  We also set up hostbased authentication which required changing the default AMI authentication procedure and then dealing with the Amazon private DNS.We then used one of the C programs we'd concocted to test CPU speed and memory.  The program creates and manipulates a large array in memory and spreads the load over the cores provided by OpenMPI.As the t1.micro AMIs simulate a single core we emulated this on our cluster by spreading the load over 3 nodes allowing only 1 core per node.  We used the 200 and 400 series, allowing for the fact that our 400 series are fairly heterogeneous when it comes to CPU speed.  Hence we distinguished between the slowest and fastest nodes available in this series.  In the graph below a higher Y axis placement is worse (slower). The t1.micro instance was faster than our 200 series (3.6GHz but relatively small CPU cache) and 400 series (1.6GHz 12MB smart cache) servers, but as we increased the number of computations the time to completion increased considerably.  Our fastest 400 series servers (2.8GHz) ultimately proved the fastest.According to the AWS site one EC2 Compute Unit provides the equivalent CPU capacity an early-2006 1.7 GHz Xeon processor.  The t1.micro instance provide a small amount of consistent CPU resources and allow you to increase CPU capacity in short bursts when additional cycles are available. This explains why for short jobs the t1.micro instances outperformed our cluster but as complexity increased the AWS scheduler decreases the relative speed of t1.micro cores.  While we'd never consider the micro instances for real HPC it does bode well for the higher end AWS products.  Issues still to be addressed though are cost per cpu cycles over time and data transfer to and from the cloud.","og_url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/","og_site_name":"UCT HPC","article_published_time":"2012-10-24T10:20:14+00:00","article_modified_time":"2015-08-14T10:19:32+00:00","og_image":[{"url":"http:\/\/blogs.uct.ac.za\/gallery\/1253\/awscomp.jpg"}],"author":"Andrew Lewis","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Andrew Lewis","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/#article","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/"},"author":{"name":"Andrew Lewis","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e"},"headline":"Poking the Amazon Coud with a stick","datePublished":"2012-10-24T10:20:14+00:00","dateModified":"2015-08-14T10:19:32+00:00","mainEntityOfPage":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/"},"wordCount":370,"publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"articleSection":["application"],"inLanguage":"en-ZA"},{"@type":"WebPage","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/","url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/","name":"Poking the Amazon Coud with a stick - UCT HPC","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/#website"},"datePublished":"2012-10-24T10:20:14+00:00","dateModified":"2015-08-14T10:19:32+00:00","breadcrumb":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/#breadcrumb"},"inLanguage":"en-ZA","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/10\/24\/poking-the-amazon-coud-with-a-stick\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucthpc.uct.ac.za\/"},{"@type":"ListItem","position":2,"name":"Poking the Amazon Coud with a stick"}]},{"@type":"WebSite","@id":"https:\/\/ucthpc.uct.ac.za\/#website","url":"https:\/\/ucthpc.uct.ac.za\/","name":"UCT HPC","description":"University of Cape Town High Performance Computing","publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-ZA"},{"@type":"Organization","@id":"https:\/\/ucthpc.uct.ac.za\/#organization","name":"University of Cape Town High Performance Computing","url":"https:\/\/ucthpc.uct.ac.za\/","logo":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/","url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","contentUrl":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","width":450,"height":423,"caption":"University of Cape Town High Performance Computing"},"image":{"@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e","name":"Andrew Lewis","image":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","caption":"Andrew Lewis"},"sameAs":["http:\/\/blogs.uct.ac.za\/blog\/big-bytes"]}]}},"_links":{"self":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/891"}],"collection":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/comments?post=891"}],"version-history":[{"count":2,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/891\/revisions"}],"predecessor-version":[{"id":2132,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/891\/revisions\/2132"}],"wp:attachment":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/media?parent=891"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/categories?post=891"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/tags?post=891"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}