{"id":881,"date":"2012-11-06T11:00:47","date_gmt":"2012-11-06T09:00:47","guid":{"rendered":"http:\/\/oldblogs.uct.ac.za\/blog\/big-bytes\/2012\/11\/06\/aws-postscript"},"modified":"2022-09-26T20:23:46","modified_gmt":"2022-09-26T18:23:46","slug":"aws-postscript","status":"publish","type":"post","link":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/","title":{"rendered":"AWS postscript"},"content":{"rendered":"<div>So after waiting 24 hours and then looking at the AWS billing reports a couple of things stand out. At first we were puzzled as to why we were billed for Run Instances as well as Spot Instances. \u00a0Turns out that when you launch a cluster the head node will be provisioned identically to the worker nodes unless you specify a different image type. The config entries below will create a cluster with 5 worker nodes and 1 head node.<\/div>\r\n<div><span style=\"font-size: xx-small;\">CLUSTER_SIZE = 6<\/span><\/div>\r\n<div><span style=\"font-size: xx-small;\">NODE_IMAGE_ID = ami-999d49f0<\/span><\/div>\r\n<div><span style=\"font-size: xx-small;\">NODE_INSTANCE_TYPE = m2.2xlarge<\/span><\/div>\r\n<div><span style=\"font-size: xx-small;\">MASTER_INSTANCE_TYPE = m1.small<\/span><\/div>\r\n<div><span style=\"font-size: xx-small;\">MASTER_IMAGE_ID = ami-899d49e0<\/span><\/div>\r\n<div>The worker nodes will be 64 bit 13xEC2 servers and the head node will be a 32 bit 1xEC2 server. \u00a0This means you won't be billed disproportionately for a very powerful head node when a smaller instance will do. \u00a0Bear in mind that the head node is not a Spot Instance only the worker nodes are so you need to factor this into your cost estimate. \u00a0Also remember that for large or very busy clusters a more powerful head node may be required.<\/div>\r\n<div>As an aside, yesterday's test came to $3.67 (about 29 ZAR).<\/div>\r\n<div><img src=\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/07\/spotcosts.jpg\" alt=\"spot costs\" border=\"0\" \/><\/div>\r\n<div>The table above indicates the rather large cost saving one can achieve by making use of Spot Instances. \u00a0This is based on our test yesterday. \u00a0The Run Instance column is what one would\u00a0<a href=\"http:\/\/aws.amazon.com\/ec2\/pricing\/\">pay for a guaranteed instance<\/a>.<\/div>","protected":false},"excerpt":{"rendered":"<div>So after waiting 24 hours and then looking at the AWS billing reports a couple of things stand out. At first we were puzzled as to why we were billed for Run Instances as well as Spot Instances. &nbsp;Turns out that when you launch a cluster the head node will be provisioned identically to the worker nodes unless you specify a different image type. The config entries below will create a cluster with 5 worker nodes and 1 head node.<\/div>\n<div><span>CLUSTER_SIZE = 6<\/span><\/div>\n<div><span>NODE_IMAGE_ID = ami-999d49f0<\/span><\/div>\n<div><span>NODE_INSTANCE_TYPE = m2.2xlarge<\/span><\/div>\n<div><span>MASTER_INSTANCE_TYPE = m1.small<\/span><\/div>\n<div><span>MASTER_IMAGE_ID = ami-899d49e0<\/span><\/div>\n<div>The worker nodes will be 64 bit 13xEC2 servers and the head node will be a 32 bit 1xEC2 server. &nbsp;This means you won&#8217;t be billed disproportionately for a very powerful head node when a smaller instance will do. &nbsp;Bear in mind that the head node is not a Spot Instance only the worker nodes are so you need to factor this into your cost estimate. &nbsp;Also remember that for large or very busy clusters a more powerful head node may be required.<\/div>\n<div>As an aside, yesterday&#8217;s test came to $3.67 (about 29 ZAR).<\/div>\n<div><img decoding=\"async\" src=\"http:\/\/blogs.uct.ac.za\/gallery\/1253\/spotcosts.jpg\" border=\"0\" alt=\"spot costs\"><\/div>\n<div>The table above indicates the rather large cost saving one can achieve by making use of Spot Instances. &nbsp;This is based on our test yesterday. &nbsp;The Run Instance column is what one would&nbsp;<a href=\"http:\/\/aws.amazon.com\/ec2\/pricing\/\">pay for a guaranteed instance<\/a>. &nbsp;<\/div>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[6],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>AWS postscript - UCT HPC<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AWS postscript - UCT HPC\" \/>\n<meta property=\"og:description\" content=\"So after waiting 24 hours and then looking at the AWS billing reports a couple of things stand out. At first we were puzzled as to why we were billed for Run Instances as well as Spot Instances. &nbsp;Turns out that when you launch a cluster the head node will be provisioned identically to the worker nodes unless you specify a different image type. The config entries below will create a cluster with 5 worker nodes and 1 head node.CLUSTER_SIZE = 6NODE_IMAGE_ID = ami-999d49f0NODE_INSTANCE_TYPE = m2.2xlargeMASTER_INSTANCE_TYPE = m1.smallMASTER_IMAGE_ID = ami-899d49e0The worker nodes will be 64 bit 13xEC2 servers and the head node will be a 32 bit 1xEC2 server. &nbsp;This means you won&#039;t be billed disproportionately for a very powerful head node when a smaller instance will do. &nbsp;Bear in mind that the head node is not a Spot Instance only the worker nodes are so you need to factor this into your cost estimate. &nbsp;Also remember that for large or very busy clusters a more powerful head node may be required.As an aside, yesterday&#039;s test came to $3.67 (about 29 ZAR).The table above indicates the rather large cost saving one can achieve by making use of Spot Instances. &nbsp;This is based on our test yesterday. &nbsp;The Run Instance column is what one would&nbsp;pay for a guaranteed instance. &nbsp;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/\" \/>\n<meta property=\"og:site_name\" content=\"UCT HPC\" \/>\n<meta property=\"article:published_time\" content=\"2012-11-06T09:00:47+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-09-26T18:23:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/07\/spotcosts.jpg\" \/>\n<meta name=\"author\" content=\"Andrew Lewis\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Andrew Lewis\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/\"},\"author\":{\"name\":\"Andrew Lewis\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\"},\"headline\":\"AWS postscript\",\"datePublished\":\"2012-11-06T09:00:47+00:00\",\"dateModified\":\"2022-09-26T18:23:46+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/\"},\"wordCount\":231,\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"articleSection\":[\"hardware\"],\"inLanguage\":\"en-ZA\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/\",\"name\":\"AWS postscript - UCT HPC\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\"},\"datePublished\":\"2012-11-06T09:00:47+00:00\",\"dateModified\":\"2022-09-26T18:23:46+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/#breadcrumb\"},\"inLanguage\":\"en-ZA\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucthpc.uct.ac.za\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AWS postscript\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"name\":\"UCT HPC\",\"description\":\"University of Cape Town High Performance Computing\",\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-ZA\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\",\"name\":\"University of Cape Town High Performance Computing\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"contentUrl\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"width\":450,\"height\":423,\"caption\":\"University of Cape Town High Performance Computing\"},\"image\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\",\"name\":\"Andrew Lewis\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"caption\":\"Andrew Lewis\"},\"sameAs\":[\"http:\/\/blogs.uct.ac.za\/blog\/big-bytes\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AWS postscript - UCT HPC","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/","og_locale":"en_US","og_type":"article","og_title":"AWS postscript - UCT HPC","og_description":"So after waiting 24 hours and then looking at the AWS billing reports a couple of things stand out. At first we were puzzled as to why we were billed for Run Instances as well as Spot Instances. &nbsp;Turns out that when you launch a cluster the head node will be provisioned identically to the worker nodes unless you specify a different image type. The config entries below will create a cluster with 5 worker nodes and 1 head node.CLUSTER_SIZE = 6NODE_IMAGE_ID = ami-999d49f0NODE_INSTANCE_TYPE = m2.2xlargeMASTER_INSTANCE_TYPE = m1.smallMASTER_IMAGE_ID = ami-899d49e0The worker nodes will be 64 bit 13xEC2 servers and the head node will be a 32 bit 1xEC2 server. &nbsp;This means you won't be billed disproportionately for a very powerful head node when a smaller instance will do. &nbsp;Bear in mind that the head node is not a Spot Instance only the worker nodes are so you need to factor this into your cost estimate. &nbsp;Also remember that for large or very busy clusters a more powerful head node may be required.As an aside, yesterday's test came to $3.67 (about 29 ZAR).The table above indicates the rather large cost saving one can achieve by making use of Spot Instances. &nbsp;This is based on our test yesterday. &nbsp;The Run Instance column is what one would&nbsp;pay for a guaranteed instance. &nbsp;","og_url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/","og_site_name":"UCT HPC","article_published_time":"2012-11-06T09:00:47+00:00","article_modified_time":"2022-09-26T18:23:46+00:00","og_image":[{"url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/07\/spotcosts.jpg"}],"author":"Andrew Lewis","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Andrew Lewis","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/#article","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/"},"author":{"name":"Andrew Lewis","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e"},"headline":"AWS postscript","datePublished":"2012-11-06T09:00:47+00:00","dateModified":"2022-09-26T18:23:46+00:00","mainEntityOfPage":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/"},"wordCount":231,"publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"articleSection":["hardware"],"inLanguage":"en-ZA"},{"@type":"WebPage","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/","url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/","name":"AWS postscript - UCT HPC","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/#website"},"datePublished":"2012-11-06T09:00:47+00:00","dateModified":"2022-09-26T18:23:46+00:00","breadcrumb":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/#breadcrumb"},"inLanguage":"en-ZA","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2012\/11\/06\/aws-postscript\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucthpc.uct.ac.za\/"},{"@type":"ListItem","position":2,"name":"AWS postscript"}]},{"@type":"WebSite","@id":"https:\/\/ucthpc.uct.ac.za\/#website","url":"https:\/\/ucthpc.uct.ac.za\/","name":"UCT HPC","description":"University of Cape Town High Performance Computing","publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-ZA"},{"@type":"Organization","@id":"https:\/\/ucthpc.uct.ac.za\/#organization","name":"University of Cape Town High Performance Computing","url":"https:\/\/ucthpc.uct.ac.za\/","logo":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/","url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","contentUrl":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","width":450,"height":423,"caption":"University of Cape Town High Performance Computing"},"image":{"@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e","name":"Andrew Lewis","image":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","caption":"Andrew Lewis"},"sameAs":["http:\/\/blogs.uct.ac.za\/blog\/big-bytes"]}]}},"_links":{"self":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/881"}],"collection":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/comments?post=881"}],"version-history":[{"count":4,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/881\/revisions"}],"predecessor-version":[{"id":4326,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/881\/revisions\/4326"}],"wp:attachment":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/media?parent=881"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/categories?post=881"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/tags?post=881"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}