{"id":863,"date":"2013-02-08T14:15:59","date_gmt":"2013-02-08T12:15:59","guid":{"rendered":"http:\/\/oldblogs.uct.ac.za\/blog\/big-bytes\/2013\/02\/08\/introducing-hex"},"modified":"2022-09-26T20:27:36","modified_gmt":"2022-09-26T18:27:36","slug":"introducing-hex","status":"publish","type":"post","link":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/","title":{"rendered":"Introducing hex"},"content":{"rendered":"
ICTS's\u00a0Next Gen HPC cluster<\/a>\u00a0consists of two\u00a0Dell C6145<\/a>\u00a0servers with 64 real \"introhex\"cores and 128GB RAM each as well as two SuperMicro servers with 4\u00a0Tesla M2090 nVidia GPUS<\/a>\u00a0each.<\/div>\r\n
The GPU servers are linked via infiniband and share an Intel SSD drive with 1.8TB of disk space on a BTRFS file system. \u00a0The servers share a common \/home folder on the NetApp and a NFS link to\u00a0hpc<\/a>'s scratch area.<\/div>\r\n
While the architecture remains similar to our original cluster we're using SLES instead of Scientific Linux. \u00a0The resource manager is\u00a0Torque 4.1.4\u00a0<\/a>and the scheduler is\u00a0Maui 3.3.1<\/a>. \u00a0These have been compiled with support for MPI, Infiniband and CUDA.<\/div>\r\n
<\/div>\r\n
We will shortly be moving the two 600 series servers in hpc.uct to the hex cluster, bringing the total core count up to 284, including the two GPU servers. \u00a0Future upgrade options include more 600 and GPU series servers as well as boosting the RAM in the 600 series from 128GB to 256GB.<\/div>","protected":false},"excerpt":{"rendered":"
ICTS’s Next Gen HPC cluster<\/a> consists of two Dell C6145<\/a> servers with 64 real cores and 128GB RAM each as well as two SuperMicro servers with 4 Tesla M2090 nVidia GPUS<\/a> each.<\/div>\n
The GPU servers are linked via infiniband and share an Intel SSD drive with 1.8TB of disk space on a BTRFS file system.  The servers share a common \/home folder on the NetApp and a NFS link to hpc<\/a>‘s scratch area.<\/div>\n
While the architecture remains similar to our original cluster we’re using SLES instead of Scientific Linux.  The resource manager is Torque 4.1.4 <\/a>and the scheduler is Maui 3.3.1<\/a>.  These have been compiled with support for MPI, Infiniband and CUDA.  <\/div>\n
\"hex\"<\/a><\/div>\n
We will shortly be moving the two 600 series servers in hpc.uct to the hex cluster, bringing the total core count up to 284, including the two GPU servers.  Future upgrade options include more 600 and GPU series servers as well as boosting the RAM in the 600 series from 128GB to 256GB. <\/div>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[],"yoast_head":"\nIntroducing hex - UCT HPC<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Introducing hex - UCT HPC\" \/>\n<meta property=\"og:description\" content=\"ICTS's Next Gen HPC cluster consists of two Dell C6145 servers with 64 real cores and 128GB RAM each as well as two SuperMicro servers with 4 Tesla M2090 nVidia GPUS each.The GPU servers are linked via infiniband and share an Intel SSD drive with 1.8TB of disk space on a BTRFS file system.  The servers share a common \/home folder on the NetApp and a NFS link to hpc's scratch area.While the architecture remains similar to our original cluster we're using SLES instead of Scientific Linux.  The resource manager is Torque 4.1.4 and the scheduler is Maui 3.3.1.  These have been compiled with support for MPI, Infiniband and CUDA.  We will shortly be moving the two 600 series servers in hpc.uct to the hex cluster, bringing the total core count up to 284, including the two GPU servers.  Future upgrade options include more 600 and GPU series servers as well as boosting the RAM in the 600 series from 128GB to 256GB. \" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/\" \/>\n<meta property=\"og:site_name\" content=\"UCT HPC\" \/>\n<meta property=\"article:published_time\" content=\"2013-02-08T12:15:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-09-26T18:27:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/08\/introhex-196x300.jpg\" \/>\n<meta name=\"author\" content=\"Andrew Lewis\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Andrew Lewis\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/\"},\"author\":{\"name\":\"Andrew Lewis\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\"},\"headline\":\"Introducing hex\",\"datePublished\":\"2013-02-08T12:15:59+00:00\",\"dateModified\":\"2022-09-26T18:27:36+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/\"},\"wordCount\":160,\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"articleSection\":[\"hpc\"],\"inLanguage\":\"en-ZA\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/\",\"name\":\"Introducing hex - UCT HPC\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\"},\"datePublished\":\"2013-02-08T12:15:59+00:00\",\"dateModified\":\"2022-09-26T18:27:36+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/#breadcrumb\"},\"inLanguage\":\"en-ZA\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucthpc.uct.ac.za\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Introducing hex\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"name\":\"UCT HPC\",\"description\":\"University of Cape Town High Performance Computing\",\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-ZA\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\",\"name\":\"University of Cape Town High Performance Computing\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"contentUrl\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"width\":450,\"height\":423,\"caption\":\"University of Cape Town High Performance Computing\"},\"image\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\",\"name\":\"Andrew Lewis\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"caption\":\"Andrew Lewis\"},\"sameAs\":[\"http:\/\/blogs.uct.ac.za\/blog\/big-bytes\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Introducing hex - UCT HPC","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/","og_locale":"en_US","og_type":"article","og_title":"Introducing hex - UCT HPC","og_description":"ICTS's Next Gen HPC cluster consists of two Dell C6145 servers with 64 real cores and 128GB RAM each as well as two SuperMicro servers with 4 Tesla M2090 nVidia GPUS each.The GPU servers are linked via infiniband and share an Intel SSD drive with 1.8TB of disk space on a BTRFS file system.  The servers share a common \/home folder on the NetApp and a NFS link to hpc's scratch area.While the architecture remains similar to our original cluster we're using SLES instead of Scientific Linux.  The resource manager is Torque 4.1.4 and the scheduler is Maui 3.3.1.  These have been compiled with support for MPI, Infiniband and CUDA.  We will shortly be moving the two 600 series servers in hpc.uct to the hex cluster, bringing the total core count up to 284, including the two GPU servers.  Future upgrade options include more 600 and GPU series servers as well as boosting the RAM in the 600 series from 128GB to 256GB. ","og_url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/","og_site_name":"UCT HPC","article_published_time":"2013-02-08T12:15:59+00:00","article_modified_time":"2022-09-26T18:27:36+00:00","og_image":[{"url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/08\/introhex-196x300.jpg"}],"author":"Andrew Lewis","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Andrew Lewis","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/#article","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/"},"author":{"name":"Andrew Lewis","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e"},"headline":"Introducing hex","datePublished":"2013-02-08T12:15:59+00:00","dateModified":"2022-09-26T18:27:36+00:00","mainEntityOfPage":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/"},"wordCount":160,"publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"articleSection":["hpc"],"inLanguage":"en-ZA"},{"@type":"WebPage","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/","url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/","name":"Introducing hex - UCT HPC","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/#website"},"datePublished":"2013-02-08T12:15:59+00:00","dateModified":"2022-09-26T18:27:36+00:00","breadcrumb":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/#breadcrumb"},"inLanguage":"en-ZA","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2013\/02\/08\/introducing-hex\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucthpc.uct.ac.za\/"},{"@type":"ListItem","position":2,"name":"Introducing hex"}]},{"@type":"WebSite","@id":"https:\/\/ucthpc.uct.ac.za\/#website","url":"https:\/\/ucthpc.uct.ac.za\/","name":"UCT HPC","description":"University of Cape Town High Performance Computing","publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-ZA"},{"@type":"Organization","@id":"https:\/\/ucthpc.uct.ac.za\/#organization","name":"University of Cape Town High Performance Computing","url":"https:\/\/ucthpc.uct.ac.za\/","logo":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/","url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","contentUrl":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","width":450,"height":423,"caption":"University of Cape Town High Performance Computing"},"image":{"@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e","name":"Andrew Lewis","image":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","caption":"Andrew Lewis"},"sameAs":["http:\/\/blogs.uct.ac.za\/blog\/big-bytes"]}]}},"_links":{"self":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/863"}],"collection":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/comments?post=863"}],"version-history":[{"count":9,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/863\/revisions"}],"predecessor-version":[{"id":4336,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/863\/revisions\/4336"}],"wp:attachment":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/media?parent=863"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/categories?post=863"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/tags?post=863"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}