{"id":1116,"date":"2011-05-10T10:27:03","date_gmt":"2011-05-10T08:27:03","guid":{"rendered":"http:\/\/oldblogs.uct.ac.za\/blog\/big-bytes\/2011\/05\/10\/parallel-poison-solution"},"modified":"2022-09-26T20:03:52","modified_gmt":"2022-09-26T18:03:52","slug":"parallel-poison-solution","status":"publish","type":"post","link":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/","title":{"rendered":"Parallel poison solution"},"content":{"rendered":"Having installed Gromacs last week we decided to run some tests to see how well it performed on our cluster.\u00a0 We downloaded a tutorial which analysed solutions of funnel web spider venom in water.\u00a0 Given the bounding block size of the virtualized water space (1 angstrom around the protein) Gromacs refused to go beyond 4 simultaneous cores as it was not possible to break this job into more then four meaningful problem sets.\r\n\r\nIt quickly became clear that with large problem sets the assistance of high speed networking would assist in achieving faster computations.\u00a0 The communication requirement between servers over a public network is fairly high, topping out at around 5% of a GB connection.\u00a0 In an environment where a large number of jobs (100+) are competing for bandwidth this would introduce significant latency.\r\n\r\n<img src=\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/07\/NIC-load.png\" alt=\"Gromacs NIC and CPU load\" border=\"0\" \/>\r\n\r\nNote that the load for the parallel openMPI jobs (the first three jobs) is lower than for the OMP jobs.\u00a0 This is because the load for OMP is kept on one server only, while openMPI distributes the load over as many servers as the job parameters allow.\u00a0 The trade off however is that the network communication between the distributed processes is high.\u00a0 Despite this the completion times between OpenMPI and OMP were fairly similar:\r\n\r\n<img src=\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/07\/GMtimes.png\" alt=\"GMtimes\" border=\"0\" \/>\r\n\r\nWe were expecting the non-openMPI time for one core to exceed the completion time for the openMPI job \"distributed\" over 1 core and were surprised to see that the times were almost identical.\u00a0 After some head scratching we realized that this was because the non-openMPI job was still a parallel job, using OMP, as the software had been compiled with this option enabled.\u00a0 In fact by not specifying the -nt parameter the job automatically grabs as many free cores as possible.\r\n\r\nGromacs data files can be exported to PDB format.\u00a0 The image below shows the final solution state after 20 pico seconds and 25000 iterations.\u00a0 The redwhite \"sticks\" show the water molecules (2 white hydrogen bonds and 1 red oxygen atom) and immersed in this is the peptide toxin of the funnel web spider, omega-Aga-IVB.\r\n\r\n<img src=\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/07\/peptide.jpg\" alt=\"spider venom peptide\" border=\"0\" \/>","protected":false},"excerpt":{"rendered":"<p>Having installed Gromacs last week we decided to run some tests to see how well it performed on our cluster.&nbsp; We downloaded a tutorial which analysed solutions of funnel web spider venom in water.&nbsp; Given the bounding block size of the virtualized water space (1 angstrom around the protein) Gromacs refused to go beyond 4 simultaneous cores as it was not possible to break this job into more then four meaningful problem sets.<\/p>\n<p>It quickly became clear that with large problem sets the assistance of high speed networking would assist in achieving faster computations.&nbsp; The communication requirement between servers over a public network is fairly high, topping out at around 5% of a GB connection.&nbsp; In an environment where a large number of jobs (100+) are competing for bandwidth this would introduce significant latency.<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/blogs.uct.ac.za\/gallery\/1253\/NIC-load.png\" border=\"0\" alt=\"Gromacs NIC and CPU load\"><\/p>\n<p>Note that the load for the parallel openMPI jobs (the first three jobs) is lower than for the OMP jobs.&nbsp; This is because the load for OMP is kept on one server only, while openMPI distributes the load over as many servers as the job parameters allow.&nbsp; The trade off however is that the network communication between the distributed processes is high.&nbsp; Despite this the completion times between OpenMPI and OMP were fairly similar:<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/blogs.uct.ac.za\/gallery\/1253\/GMtimes.png\" border=\"0\" alt=\"GMtimes\"><\/p>\n<p>We were expecting the non-openMPI time for one core to exceed the completion time for the openMPI job &#8220;distributed&#8221; over 1 core and were surprised to see that the times were almost identical.&nbsp; After some head scratching we realized that this was because the non-openMPI job was still a parallel job, using OMP, as the software had been compiled with this option enabled.&nbsp; In fact by not specifying the -nt parameter the job automatically grabs as many free cores as possible.<\/p>\n<p>Gromacs data files can be exported to PDB format.&nbsp; The image below shows the final solution state after 20 pico seconds and 25000 iterations.&nbsp; The redwhite &#8220;sticks&#8221; show the water molecules (2 white hydrogen bonds and 1 red oxygen atom) and immersed in this is the peptide toxin of the funnel web spider, omega-Aga-IVB.<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/blogs.uct.ac.za\/gallery\/1253\/peptide.jpg\" border=\"0\" alt=\"spider venom peptide\"><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9,10],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Parallel poison solution - UCT HPC<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Parallel poison solution - UCT HPC\" \/>\n<meta property=\"og:description\" content=\"Having installed Gromacs last week we decided to run some tests to see how well it performed on our cluster.&nbsp; We downloaded a tutorial which analysed solutions of funnel web spider venom in water.&nbsp; Given the bounding block size of the virtualized water space (1 angstrom around the protein) Gromacs refused to go beyond 4 simultaneous cores as it was not possible to break this job into more then four meaningful problem sets.It quickly became clear that with large problem sets the assistance of high speed networking would assist in achieving faster computations.&nbsp; The communication requirement between servers over a public network is fairly high, topping out at around 5% of a GB connection.&nbsp; In an environment where a large number of jobs (100+) are competing for bandwidth this would introduce significant latency.Note that the load for the parallel openMPI jobs (the first three jobs) is lower than for the OMP jobs.&nbsp; This is because the load for OMP is kept on one server only, while openMPI distributes the load over as many servers as the job parameters allow.&nbsp; The trade off however is that the network communication between the distributed processes is high.&nbsp; Despite this the completion times between OpenMPI and OMP were fairly similar:We were expecting the non-openMPI time for one core to exceed the completion time for the openMPI job &quot;distributed&quot; over 1 core and were surprised to see that the times were almost identical.&nbsp; After some head scratching we realized that this was because the non-openMPI job was still a parallel job, using OMP, as the software had been compiled with this option enabled.&nbsp; In fact by not specifying the -nt parameter the job automatically grabs as many free cores as possible.Gromacs data files can be exported to PDB format.&nbsp; The image below shows the final solution state after 20 pico seconds and 25000 iterations.&nbsp; The redwhite &quot;sticks&quot; show the water molecules (2 white hydrogen bonds and 1 red oxygen atom) and immersed in this is the peptide toxin of the funnel web spider, omega-Aga-IVB.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/\" \/>\n<meta property=\"og:site_name\" content=\"UCT HPC\" \/>\n<meta property=\"article:published_time\" content=\"2011-05-10T08:27:03+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-09-26T18:03:52+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/07\/NIC-load.png\" \/>\n<meta name=\"author\" content=\"Andrew Lewis\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Andrew Lewis\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/\"},\"author\":{\"name\":\"Andrew Lewis\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\"},\"headline\":\"Parallel poison solution\",\"datePublished\":\"2011-05-10T08:27:03+00:00\",\"dateModified\":\"2022-09-26T18:03:52+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/\"},\"wordCount\":337,\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"articleSection\":[\"application\",\"MPI\"],\"inLanguage\":\"en-ZA\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/\",\"name\":\"Parallel poison solution - UCT HPC\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\"},\"datePublished\":\"2011-05-10T08:27:03+00:00\",\"dateModified\":\"2022-09-26T18:03:52+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/#breadcrumb\"},\"inLanguage\":\"en-ZA\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucthpc.uct.ac.za\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Parallel poison solution\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"name\":\"UCT HPC\",\"description\":\"University of Cape Town High Performance Computing\",\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-ZA\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\",\"name\":\"University of Cape Town High Performance Computing\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"contentUrl\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"width\":450,\"height\":423,\"caption\":\"University of Cape Town High Performance Computing\"},\"image\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e\",\"name\":\"Andrew Lewis\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g\",\"caption\":\"Andrew Lewis\"},\"sameAs\":[\"http:\/\/blogs.uct.ac.za\/blog\/big-bytes\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Parallel poison solution - UCT HPC","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/","og_locale":"en_US","og_type":"article","og_title":"Parallel poison solution - UCT HPC","og_description":"Having installed Gromacs last week we decided to run some tests to see how well it performed on our cluster.&nbsp; We downloaded a tutorial which analysed solutions of funnel web spider venom in water.&nbsp; Given the bounding block size of the virtualized water space (1 angstrom around the protein) Gromacs refused to go beyond 4 simultaneous cores as it was not possible to break this job into more then four meaningful problem sets.It quickly became clear that with large problem sets the assistance of high speed networking would assist in achieving faster computations.&nbsp; The communication requirement between servers over a public network is fairly high, topping out at around 5% of a GB connection.&nbsp; In an environment where a large number of jobs (100+) are competing for bandwidth this would introduce significant latency.Note that the load for the parallel openMPI jobs (the first three jobs) is lower than for the OMP jobs.&nbsp; This is because the load for OMP is kept on one server only, while openMPI distributes the load over as many servers as the job parameters allow.&nbsp; The trade off however is that the network communication between the distributed processes is high.&nbsp; Despite this the completion times between OpenMPI and OMP were fairly similar:We were expecting the non-openMPI time for one core to exceed the completion time for the openMPI job \"distributed\" over 1 core and were surprised to see that the times were almost identical.&nbsp; After some head scratching we realized that this was because the non-openMPI job was still a parallel job, using OMP, as the software had been compiled with this option enabled.&nbsp; In fact by not specifying the -nt parameter the job automatically grabs as many free cores as possible.Gromacs data files can be exported to PDB format.&nbsp; The image below shows the final solution state after 20 pico seconds and 25000 iterations.&nbsp; The redwhite \"sticks\" show the water molecules (2 white hydrogen bonds and 1 red oxygen atom) and immersed in this is the peptide toxin of the funnel web spider, omega-Aga-IVB.","og_url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/","og_site_name":"UCT HPC","article_published_time":"2011-05-10T08:27:03+00:00","article_modified_time":"2022-09-26T18:03:52+00:00","og_image":[{"url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/07\/NIC-load.png"}],"author":"Andrew Lewis","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Andrew Lewis","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/#article","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/"},"author":{"name":"Andrew Lewis","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e"},"headline":"Parallel poison solution","datePublished":"2011-05-10T08:27:03+00:00","dateModified":"2022-09-26T18:03:52+00:00","mainEntityOfPage":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/"},"wordCount":337,"publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"articleSection":["application","MPI"],"inLanguage":"en-ZA"},{"@type":"WebPage","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/","url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/","name":"Parallel poison solution - UCT HPC","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/#website"},"datePublished":"2011-05-10T08:27:03+00:00","dateModified":"2022-09-26T18:03:52+00:00","breadcrumb":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/#breadcrumb"},"inLanguage":"en-ZA","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2011\/05\/10\/parallel-poison-solution\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucthpc.uct.ac.za\/"},{"@type":"ListItem","position":2,"name":"Parallel poison solution"}]},{"@type":"WebSite","@id":"https:\/\/ucthpc.uct.ac.za\/#website","url":"https:\/\/ucthpc.uct.ac.za\/","name":"UCT HPC","description":"University of Cape Town High Performance Computing","publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-ZA"},{"@type":"Organization","@id":"https:\/\/ucthpc.uct.ac.za\/#organization","name":"University of Cape Town High Performance Computing","url":"https:\/\/ucthpc.uct.ac.za\/","logo":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/","url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","contentUrl":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","width":450,"height":423,"caption":"University of Cape Town High Performance Computing"},"image":{"@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/c183ad1c0a1063124a72d63963ae9c7e","name":"Andrew Lewis","image":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9652c9c73beeab594b8dc2383a880048?s=96&d=mm&r=g","caption":"Andrew Lewis"},"sameAs":["http:\/\/blogs.uct.ac.za\/blog\/big-bytes"]}]}},"_links":{"self":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/1116"}],"collection":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/comments?post=1116"}],"version-history":[{"count":4,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/1116\/revisions"}],"predecessor-version":[{"id":4287,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/1116\/revisions\/4287"}],"wp:attachment":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/media?parent=1116"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/categories?post=1116"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/tags?post=1116"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}