{"id":1073,"date":"2011-07-28T15:32:04","date_gmt":"2011-07-28T13:32:04","guid":{"rendered":"http:\/\/oldblogs.uct.ac.za\/blog\/big-bytes\/2011\/07\/28\/mpi-standardization"},"modified":"2015-08-14T13:23:21","modified_gmt":"2015-08-14T11:23:21","slug":"mpi-standardization","status":"publish","type":"post","link":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/","title":{"rendered":"MPI standardization"},"content":{"rendered":"Our cluster is a heterogenous mixture of 3 blade types and 2 operating sytems, the latter being Scientific Linux 5.4 and 5.5.\u00a0 Unfortunately these two versions of OS come with slightly differing versions of openmpi.\u00a0 In order to allow jobs to span all blade architectures we have bypassed SL's install of openmpi and upgraded it manually to 1.4-4.el5.\r\n\r\nPros: Users can now create a hostfile referencing all hosts in the 200, 300 and 400 series.\u00a0 This will allow jobs to span up to 96 cores.\u00a0 Run mpi-selector-menu from the CLI to select the installed version of openmpi.\r\n\r\nCons: Memory size and CPU speeds differ between series which will cause a discrepancy in the completion times depending on the critical resource (RAM or MHz) defined by a job's algorithm.\u00a0 Until all threads are finished all nodes will be marked as in use.\r\n\r\n&nbsp;","protected":false},"excerpt":{"rendered":"<p>Our cluster is a heterogenous mixture of 3 blade types and 2 operating sytems, the latter being Scientific Linux 5.4 and 5.5.&nbsp; Unfortunately these two versions of OS come with slightly differing versions of openmpi.&nbsp; In order to allow jobs to span all blade architectures we have bypassed SL&#8217;s install of openmpi and upgraded it manually to 1.4-4.el5.<\/p>\n<p>Pros: Users can now create a hostfile referencing all hosts in the 200, 300 and 400 series.&nbsp; This will allow jobs to span up to 96 cores.&nbsp; Run mpi-selector-menu from the CLI to select the installed version of openmpi.<\/p>\n<p>Cons: Memory size and CPU speeds differ between series which will cause a discrepancy in the completion times depending on the critical resource (RAM or MHz) defined by a job&#8217;s algorithm.&nbsp; Until all threads are finished all nodes will be marked as in use.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[10],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>MPI standardization - UCT HPC<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"MPI standardization - UCT HPC\" \/>\n<meta property=\"og:description\" content=\"Our cluster is a heterogenous mixture of 3 blade types and 2 operating sytems, the latter being Scientific Linux 5.4 and 5.5.&nbsp; Unfortunately these two versions of OS come with slightly differing versions of openmpi.&nbsp; In order to allow jobs to span all blade architectures we have bypassed SL&#039;s install of openmpi and upgraded it manually to 1.4-4.el5.Pros: Users can now create a hostfile referencing all hosts in the 200, 300 and 400 series.&nbsp; This will allow jobs to span up to 96 cores.&nbsp; Run mpi-selector-menu from the CLI to select the installed version of openmpi.Cons: Memory size and CPU speeds differ between series which will cause a discrepancy in the completion times depending on the critical resource (RAM or MHz) defined by a job&#039;s algorithm.&nbsp; Until all threads are finished all nodes will be marked as in use.&nbsp;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/\" \/>\n<meta property=\"og:site_name\" content=\"UCT HPC\" \/>\n<meta property=\"article:published_time\" content=\"2011-07-28T13:32:04+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2015-08-14T11:23:21+00:00\" \/>\n<meta name=\"author\" content=\"Andrew Lewis\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Andrew Lewis\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/\"},\"author\":{\"name\":\"Andrew Lewis\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\"},\"headline\":\"MPI standardization\",\"datePublished\":\"2011-07-28T13:32:04+00:00\",\"dateModified\":\"2015-08-14T11:23:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/\"},\"wordCount\":137,\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"articleSection\":[\"MPI\"],\"inLanguage\":\"en-ZA\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/\",\"name\":\"MPI standardization - UCT HPC\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\"},\"datePublished\":\"2011-07-28T13:32:04+00:00\",\"dateModified\":\"2015-08-14T11:23:21+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/#breadcrumb\"},\"inLanguage\":\"en-ZA\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucthpc.uct.ac.za\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"MPI standardization\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"name\":\"UCT HPC\",\"description\":\"University of Cape Town High Performance Computing\",\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-ZA\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\",\"name\":\"University of Cape Town High Performance Computing\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"contentUrl\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"width\":450,\"height\":423,\"caption\":\"University of Cape Town High Performance Computing\"},\"image\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\",\"name\":\"Andrew Lewis\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"caption\":\"Andrew Lewis\"},\"sameAs\":[\"http:\/\/blogs.uct.ac.za\/blog\/big-bytes\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"MPI standardization - UCT HPC","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/","og_locale":"en_US","og_type":"article","og_title":"MPI standardization - UCT HPC","og_description":"Our cluster is a heterogenous mixture of 3 blade types and 2 operating sytems, the latter being Scientific Linux 5.4 and 5.5.&nbsp; Unfortunately these two versions of OS come with slightly differing versions of openmpi.&nbsp; In order to allow jobs to span all blade architectures we have bypassed SL's install of openmpi and upgraded it manually to 1.4-4.el5.Pros: Users can now create a hostfile referencing all hosts in the 200, 300 and 400 series.&nbsp; This will allow jobs to span up to 96 cores.&nbsp; Run mpi-selector-menu from the CLI to select the installed version of openmpi.Cons: Memory size and CPU speeds differ between series which will cause a discrepancy in the completion times depending on the critical resource (RAM or MHz) defined by a job's algorithm.&nbsp; Until all threads are finished all nodes will be marked as in use.&nbsp;","og_url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/","og_site_name":"UCT HPC","article_published_time":"2011-07-28T13:32:04+00:00","article_modified_time":"2015-08-14T11:23:21+00:00","author":"Andrew Lewis","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Andrew Lewis","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/#article","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/"},"author":{"name":"Andrew Lewis","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e"},"headline":"MPI standardization","datePublished":"2011-07-28T13:32:04+00:00","dateModified":"2015-08-14T11:23:21+00:00","mainEntityOfPage":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/"},"wordCount":137,"publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"articleSection":["MPI"],"inLanguage":"en-ZA"},{"@type":"WebPage","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/","url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/","name":"MPI standardization - UCT HPC","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/#website"},"datePublished":"2011-07-28T13:32:04+00:00","dateModified":"2015-08-14T11:23:21+00:00","breadcrumb":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/#breadcrumb"},"inLanguage":"en-ZA","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/07\/28\/mpi-standardization\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucthpc.uct.ac.za\/"},{"@type":"ListItem","position":2,"name":"MPI standardization"}]},{"@type":"WebSite","@id":"https:\/\/ucthpc.uct.ac.za\/#website","url":"https:\/\/ucthpc.uct.ac.za\/","name":"UCT HPC","description":"University of Cape Town High Performance Computing","publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-ZA"},{"@type":"Organization","@id":"https:\/\/ucthpc.uct.ac.za\/#organization","name":"University of Cape Town High Performance Computing","url":"https:\/\/ucthpc.uct.ac.za\/","logo":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/","url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","contentUrl":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","width":450,"height":423,"caption":"University of Cape Town High Performance Computing"},"image":{"@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e","name":"Andrew Lewis","image":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","caption":"Andrew Lewis"},"sameAs":["http:\/\/blogs.uct.ac.za\/blog\/big-bytes"]}]}},"_links":{"self":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/1073"}],"collection":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/comments?post=1073"}],"version-history":[{"count":2,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/1073\/revisions"}],"predecessor-version":[{"id":2222,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/1073\/revisions\/2222"}],"wp:attachment":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/media?parent=1073"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/categories?post=1073"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/tags?post=1073"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}