Slidecast: Cycle Computing Runs Amazing 10,600 Instance on AWS for Big Pharma


In this slidecast, Jason Stowe from Cycle Computing describes how the company spun up a 10,600 instance HPC cluster in 2 hours with CycleCloud on Amazon EC2. Using just one Chef 11 server and one purpose in mind, this on-the-fly cluster was used to accelerate life science research relating to a cancer target for a Big 10 Pharmaceutical company.

To tackle this problem, we decided to build software to create a CycleCloud utility supercomputer from 10,600 cloud instances, each of which was a multi-core machine! This makes this cluster the largest server-count cloud HPC environment that we know about, or has been made public to date (the former utility supercomputing leader was our 6,732 instance cluster for Sch√∂dinger from 2012). If this cluster were a physical environment, analysts said it would occupy a 12,000 sq ft data center space, costing $44 million. Instead, we created this in 2 hours, with these 10,600 hosts, used it for 9 more, at a peak cost of $549.72 per hour, and turned it off for a total cost of $4,362. Wow!”

Read the Full Story * Download the MP3 * Download the Slides (PDF) * Subscribe on iTunes * If Dropbox is blocked, download audio from Google Drive.