The options for High Performance Computing (HPC) systems can be overwhelming with the different expenses and benefits associated with each system. The different systems currently available fall into the following categories: on-premise, cloud-enabled (full or hybrid), and bare-metal cloud.

Depending on your current and future HPC and organizational demands, each system offers benefits and limitations that need to be defined and compared. One of the main comparisons between systems is usually the Total Cost of Ownership (TCO). As I mentioned in a previous blog post, TCO not exactly a good fit for making buying decisions between fundamentally dissimilar alternatives. The TCO of on-premise HPC systems has been discussed for +30 years, even by our VP of Sales in his blog “The Real Cost of High-Performance Computing.” For people who are considering buying on-premise HPC systems, there are some hidden expenses that are often overlooked when calculating the TCO of and on-premise HPC system.

In this post, I intend to break down the TCO of an on-premise system and expose some expenses that may be overlooked.

A quick review on TCO

The broad definition of an on-premise HPC system’s TCO is that you sum the amount of all direct and indirect expenses correlated with your prospective system. The more obvious expenses are hardware, software, staffing, and power.  For hardware, you need the following: servers, wiring, ToR switches, aggregation switches, server racks, power distribution units, etc. Then you must buy software that coordinates the communication between each node to solve complex problems. In addition, you must buy licenses for the software you plan on using. A resource that can be extremely variable and hard to estimate is the staffing required to develop, deploy, and maintain the on-premise HPC system. Finally, on-premise HPC systems require a lot of power and cooling capabilities: it is essential to calculate your energy consumption and how it will affect your operational expenses. Take the sum of the expenses for the items above and you have the basic TCO for your on-premise HPC system; however, there are some hidden costs that can heavily affect the TCO of your on-premise system.  

Real-world, Hidden Costs

#1 The facilities hosting your HPC systems have cost dependencies that reach further than at first glance. Ensuring your facility has the proper cooling and power provisions necessary to support the current system and its potential scalability can save a lot of expenses down the road. Power is a major expense and can be extremely impactful on your overall operating expense. Depending on cluster location and utilization, your power costs can vary greatly. Due to your location, you may also see highly variable power prices that will heavily affect how you operate your HPC system to minimize expenses. In some cases, power can become over 1/3 of your operating expenses. Facilities and energy are important to consider when calculating your TCO and, for a large facility, should be considered a primary concern.

#2 Staffing will cost and vary more than you think, with performance and uptime suffering if neglected. One of the most variable and elusive expenses to define is the staffing for on-premise HPC systems. It can be very difficult to find, hire and train good Operations and IT Managers that can perform the development, deployment, and maintenance of an HPC system. Designing an HPC system requires expensive specialists to match the best hardware and software for your computing demands. The procurement of the system alone can cost as much as 5% of the total HPC system and takes at least 6 months. During this time, you must continue paying specialists to assemble the cluster while receiving no reward for the HPC system. Once deployed, the systems require very specific IT staffing to ensure its’ maintenance and operation. These employees require specialized skills to test and protect your HPC system’s longevity and performance. Finding the right employees to perform these functions can be cumbersome and costly, but is a priority when considering deploying an on-premise HPC system.

#3 Underutilization costs more than just the idle time, the associated overhead is substantial as well. An idle HPC system not only lowers your ROI, but can have devastating impacts on your product development cycle. Back-up systems can be overlooked because they are not considered necessary expenses to have an operating HPC system; however, the consequences for not having them can be dire. Generators, switches, gas, and maintenance of your backup energy system are all necessary to ensure that your systems are protected from power outages. Comparable to back-up energy provisions, back-up hardware is extremely important to mitigate an idle HPC system. Spare hardware is important to have on hand in case there is an issue; without backup hardware, you can find your system sitting idle while the part is repaired or bought. If you fail to plan, you should plan to fail; this is especially true for running an on-premise HPC system.  

#4 Finally, on-premise technology is a constant uphill (and usually losing) battle. This is the harm caused by not utilized the best technology, and having to spend enormous efforts and capital to race to keep up. When comparing HPC systems, you have to acknowledge the costs and rewards, and their effect on each other. Not using the best technology can create expenses that stem from forfeiting rewards that are given by the best system. The expenses correlated to not using the best HPC solution are: lost productivity, missed innovation, longer time-to-solution, technology refresh cost, IT risk management, and increased IT debt and commitment. The most harmful forfeited reward is inefficiency in the research pipeline which creates a plethora of expenses correlated to the increase in time-to-market, delay in innovation, and increase in researcher idle time. The lack of HPC technology can cause your organization to have irreparable implications such as not being able to research larger problems and missing innovations that can make your organization uncompetitive. These expenses are often difficult to calculate because you have to assess how much more efficient your team will be with a better HPC solution and then work backwards to calculate the expenses correlated to inefficiency.  

In summary, finding the true TCO of an on-premise HPC system can prove very difficult when considering all the hidden costs: staffing, facilities, power consumption, backup provisions, and forfeited rewards. I argue that one of the most important expenses to consider when comparing HPC systems is the expenses caused by forfeited rewards; however, these prove to be the most difficult to calculate and predict. The topic of TCO comparisons between cloud-enabled and on-premise HPC systems has been discussed regularly and is still not clearly defined. It is a comparison that we are working to improve, so if you have any comments or questions on this blog post or TCO, we would love to hear what you think.

Sara Jeanes. (2017, June 19). Cloud vs. Datacenter Costs for High Performance Computing (HPC): A Real World Example. Retrieved from: https://www.internet2.edu/blogs/detail/14114

Tony Spagnuolo. (2015, January). The Real Cost of High Performance Computing. Retrieved from: https://blog.rescale.com/the-real-cost-of-high-performance-computing/

Wolfgang Gentzsch. (2016, March 6). A Total Cost Analysis for Manufacturers of In-house Computing Resources and Cloud Computing. Retrieved from: https://community.theubercloud.com/wp-content/uploads/2016/04/TCO-Study-UberCloud.pdf

This article was written by Thomas Helmonds.

Total Cost of Ownership (TCO) is a powerful financial tool that allows you to understand the direct and indirect expenses related to an asset, such as your HPC system. Calculating the TCO for an on-premise HPC system is direct: add up all expenses related to your system and its management for the entirety of its deployment. But what happens when you’re interested in switching to a cloud-enabled HPC environment? Can you confidently compare the cloud-enabled HPC system’s TCO with an on-premise HPC system’s TCO?

This question has been addressed by many different institutions.

Our view is simple: TCO is a poor financial tool for evaluating the value of cloud-enabled HPC. Comparing a system with a static environment against a dynamic environment creates an unreliable and misleading analysis. It is an apples to oranges comparison, and using TCO to assess cloud-enabled HPC attempts to make apple juice from oranges.

What is a static environment and how does it apply to my TCO analysis?

Static environments for TCO are used when you have set expense for a set return. For an on-premise system, you can get X amount of computing power for Y amount of dollars. This same relationship goes on for most expenses in the cost analysis of an on-premise HPC system until you reach a comprehensive TCO. There are some variable costs involved (fluctuation in software pricing, staffing, energy, unpredicted errors, etc.); however, margins can be used to monitor their influence on the TCO. Essentially, you end up with the general TCO analysis of X computing power = Y expenses ± margin of change. This is a great tool for comparing systems with little expense variations and known rewards that create a near-linear relationship. However, what happens when the computing power is nearly infinite, and the expenses are reactive, as is the case for cloud computing?

What is a dynamic environment and how does it apply to my TCO analysis?

A dynamic environment for a TCO analysis is a system where the expenses and rewards are not directly correlated, making them difficult to define and compare. In a cloud-enabled HPC system, you pay for computing power when you need it; there is little initial capital expenditure needed to use cloud-enabled HPC, when compared to on-premise HPC systems. In this environment, your expenses for HPC become less predictable and more reactive because they are generated from your computing demand. In addition, you are no longer constrained by a set capacity or architecture of computing resources, so your reward is extremely variable due to how you utilize HPC. This scalability and agility can heavily influence your HPC usage; especially if your current system is inhibiting your simulation-throughput and potential Design of Experiment (DOE). The rewards of cloud computing beckon the question: if you have less restrictions on HPC, would you utilize it differently?

What happens when you use TCO to compare on-premise vs cloud-enabled HPC systems?

TCO is a tool that is helpful for static environments, but when you try to take the same static tool and apply it to a highly dynamic environment, it is misleading. For example, consider you want to calculate the TCO of an on-premise HPC system. First, you must predict your peak usage and utilization for a system that will be used for approximately 3-5 years. To manage all an organization’s requirements, trade offs are made between peak capacity and the cost of obsolescence. Then you must pay the massive initial capital expenditure to purchase all the hardware, software, and staff required to assemble and operate the system. Calculate all these expenses and you receive your TCO for a system that awards you limited computing resources.

Now, try to use the same analysis of a cloud-enabled HPC system. Most take the projected peak computing power and average utilization and multiply it by the price to compute in their prospective cloud service provider. This is the first problem, you’re already treating both systems as if their rewards and expenses are equal. With cloud-enabled HPC systems, you have instant access to the latest hardware and applications which means you are always utilizing the best infrastructure for your workflow. By utilizing cloud resources, your computing power becomes near-infinite, meaning there is no reason to have a queue for running simulations, which increases your productivity. The limitless and diverse computing resources allows for innovations in the research and design process that are essential to getting better products to market before competitors. The inability to easily scale and upgrade resources for an on-premise HPC system can severely inhibit your ability to compete. The differences in rewards makes it hard to quantify the expenses associated with the aging on-premise HPC system’s effect on potential new workflows that can help you out-compete your competition.

When comparing HPC solution’s TCO, you must acknowledge the rewards provided by each solution, because the lack of a rewards should be reflected as an expense in the competitor’s TCO. For example, if your cloud computing solution provides no queue time, better computing performance, and new DOEs, but your on-premise solution doesn’t, then you must calculate the expenses of inefficiency correlated to the absence of rewards from the on-premise system. That is the only way to level the TCO with the corresponding rewards, but it proves extremely difficult to define exact numbers for each reward; henceforth, making TCO a misleading and inaccurate tool. Comparing the TCO and rewards of cloud-enabled and on-premise HPC systems is pointless because the tool does not address the reality of each system; one is static and requires massive investment to create limited computing power, and the other is agile and requires pay-as-you-go expenses for limitless computing power.

Determining the financial implications of incorporating cloud-enabled HPC into you HPC system can be difficult. Thankfully, Rescale has many specialists and confidential tools to help define the benefit of cloud-enabled HPC on your organization.

Come talk to us today.

This article was written by Thomas Helmonds.

Accelerating innovation and powering big compute at hundreds of leading enterprises

With the year coming to a close, I’m excited to share the progress we’ve made at Rescale—not only as the leading Big Compute and cloud HPC solution in the market, but also as one of the fastest growing enterprise software companies of 2017. It’s been a busy year as we’ve landed over 100 new enterprise customers, fueling our rapid growth in multiple key industry verticals including aerospace, automotive, life sciences, universities and with broad geographic coverage throughout the Americas, Europe, and Asia.

Our customers are deeply engaging with the ScaleX Enterprise platform and our portfolio of industry solutions.  Usage of Rescale has grown by 30% month-over-month throughout the entire year, resulting in numerous Silicon Valley investors identifying Rescale as the fastest growing enterprise software company in 2017.  Customers are delivering incredible results accelerating their innovation capabilities, including: Airbus accelerating development of complex aerodynamics, LSIS deploying the first cloud-based engineering environment in Korea, RWDI implementing hybrid HPC across their enterprise, Boom Supersonic building the world’s next-generation supersonic jetliner, and many more.

Continue reading

This article was written by Joris Poort.

This article originally appeared on ENGINEERING.com. To see the full article, click here.

The concept of the digital twin brings versatility to the engineering world. By creating a virtual representation of a product, engineers can investigate designs to further product development, in-service optimizations and quality assurance.

Digital twin

Digital twins help engineers predict real world behavior, optimize designs and gain new insights and understandings of use cases. (Image courtesy of Rescale.)

Now with the implementation of the Internet of Things (IoT), the potential of the digital twin grows.

Engineers can link real-time data into their digital mock-ups, allowing for better understandings of the physical world.

However, digital twins don’t just come off the shelf. Since every modeled behavior is built in different system, engineering and IT teams experience a considerable challenge linking their disparate tools into one model. Joris Poort, CEO of Rescale, explained that this is where vendor-agnostic cloud HPC, like Rescale, can really shine.

To continue reading, please read the full article on ENGINEERING.com.

This article was written by Shawn Wasserman, ENGINEERING.com.