In this Big Compute podcast episode, host Gabriel Broner interviews Mike Woodacre, HPE Fellow, to discuss the shift from CPUs to an emerging diversity of architectures. Hear about the evolution of CPUs, the advent of GPUs with increasing data parallelism, memory driven computing, and the potential benefits of a cloud environment with access to multiple architectures.
After many years of CPU architectures doubling in performance every eighteen months we have reached a plateau due to constraints of miniaturization and power. This has led to a new era of diversity with systems that use CPUs, GPUs, FPGAs, etc. Woodacre welcomes the new diversity after years of consolidation and commoditization:
“It is a really exciting time in the industry. We are going back to the variety we used to have. Because volumes and commoditization, we gravitated to a few variants. As we started hitting the limits of the ability to scale we are seeing a new era of specialization. At a time data continue to grow exponentially, there are new business reasons to develop unique architectures. The challenge is to pick the right tool for the job.”
The Evolution of CPUs
CPUs have evolved in the last few generations. They have increased the number of cores, and they have increased its memory bandwidth.
“The biggest change has been the continuing growing number of cores. The challenge is how you feed them. You need more memory bandwidth, I/O.”
“When you analyze HPC applications, pretty soon memory bandwidth is the limiting factor. Skylake can go from 4 cores to 28 cores, but in some benchmarks you may top out at 12 cores because of memory bandwidth.”
GPUs, which initially started as the graphical processing unit are, according to Woodacre, “the ultimate data parallel device” which has found success in AI/machine learning/deep learning:
“Deep learning at the core is a matrix multiplication, and you can take advantage of the multiple multiplication units to perform training in hours, not weeks.”
Memory Driven Architectures
Emerging memory driven architectures hold all data in memory and they can accelerate insights and make us rethink workflows:
“Data continues to grow exponentially and traditional architectures cannot cope. By having data in memory you remove the storage bottlenecks and speed up the analysis work. You can take the current pipeline of preprocessing, simulation, analysis, and by keeping data in memory achieve tremendous speedups.”
Cloud as an Enabler
Cloud helps enable this new diversity of architectures. As each application and workflow benefit from a different architecture, cloud allows different applications to run on different architectures to effectively use the right tool for the job.
“Security of cloud used to be a concern but it’s now pretty much in the background. The public cloud provides an entry point to explore all these architectural options. If the software applications and licensing is provided, that helps people with the on-ramp. In the end you have to decide what makes the most business sense for you.”
Mike Woodacre is a Fellow at HPE. Over the years Mike has been a systems architect and Chief Engineer at MIPS, SGI and HPE, where he set direction for hardware architectures.
Gabriel Broner is VP & GM of HPC at Rescale. Prior to joining Rescale in July 2017, Gabriel spent 25 years in the industry as OS architect at Cray, GM at Microsoft, head of innovation at Ericsson, and VP & GM of HPC at SGI/HPE.
The options for High Performance Computing (HPC) systems can be overwhelming with the different expenses and benefits associated with each system. The different systems currently available fall into the following categories: on-premise, cloud-enabled (full or hybrid), and bare-metal cloud.
In this post, I intend to break down the TCO of an on-premise system and expose some expenses that may be overlooked.
A quick review on TCO
The broad definition of an on-premise HPC system’s TCO is that you sum the amount of all direct and indirect expenses correlated with your prospective system. The more obvious expenses are hardware, software, staffing, and power. For hardware, you need the following: servers, wiring, ToR switches, aggregation switches, server racks, power distribution units, etc. Then you must buy software that coordinates the communication between each node to solve complex problems. In addition, you must buy licenses for the software you plan on using. A resource that can be extremely variable and hard to estimate is the staffing required to develop, deploy, and maintain the on-premise HPC system. Finally, on-premise HPC systems require a lot of power and cooling capabilities: it is essential to calculate your energy consumption and how it will affect your operational expenses. Take the sum of the expenses for the items above and you have the basic TCO for your on-premise HPC system; however, there are some hidden costs that can heavily affect the TCO of your on-premise system.
Real-world, Hidden Costs
#1 The facilities hosting your HPC systems have cost dependencies that reach further than at first glance. Ensuring your facility has the proper cooling and power provisions necessary to support the current system and its potential scalability can save a lot of expenses down the road. Power is a major expense and can be extremely impactful on your overall operating expense. Depending on cluster location and utilization, your power costs can vary greatly. Due to your location, you may also see highly variable power prices that will heavily affect how you operate your HPC system to minimize expenses. In some cases, power can become over 1/3 of your operating expenses. Facilities and energy are important to consider when calculating your TCO and, for a large facility, should be considered a primary concern.
#2 Staffing will cost and vary more than you think, with performance and uptime suffering if neglected. One of the most variable and elusive expenses to define is the staffing for on-premise HPC systems. It can be very difficult to find, hire and train good Operations and IT Managers that can perform the development, deployment, and maintenance of an HPC system. Designing an HPC system requires expensive specialists to match the best hardware and software for your computing demands. The procurement of the system alone can cost as much as 5% of the total HPC system and takes at least 6 months. During this time, you must continue paying specialists to assemble the cluster while receiving no reward for the HPC system. Once deployed, the systems require very specific IT staffing to ensure its’ maintenance and operation. These employees require specialized skills to test and protect your HPC system’s longevity and performance. Finding the right employees to perform these functions can be cumbersome and costly, but is a priority when considering deploying an on-premise HPC system.
#3 Underutilization costs more than just the idle time, the associated overhead is substantial as well. An idle HPC system not only lowers your ROI, but can have devastating impacts on your product development cycle. Back-up systems can be overlooked because they are not considered necessary expenses to have an operating HPC system; however, the consequences for not having them can be dire. Generators, switches, gas, and maintenance of your backup energy system are all necessary to ensure that your systems are protected from power outages. Comparable to back-up energy provisions, back-up hardware is extremely important to mitigate an idle HPC system. Spare hardware is important to have on hand in case there is an issue; without backup hardware, you can find your system sitting idle while the part is repaired or bought. If you fail to plan, you should plan to fail; this is especially true for running an on-premise HPC system.
#4 Finally, on-premise technology is a constant uphill (and usually losing) battle. This is the harm caused by not utilized the best technology, and having to spend enormous efforts and capital to race to keep up. When comparing HPC systems, you have to acknowledge the costs and rewards, and their effect on each other. Not using the best technology can create expenses that stem from forfeiting rewards that are given by the best system. The expenses correlated to not using the best HPC solution are: lost productivity, missed innovation, longer time-to-solution, technology refresh cost, IT risk management, and increased IT debt and commitment. The most harmful forfeited reward is inefficiency in the research pipeline which creates a plethora of expenses correlated to the increase in time-to-market, delay in innovation, and increase in researcher idle time. The lack of HPC technology can cause your organization to have irreparable implications such as not being able to research larger problems and missing innovations that can make your organization uncompetitive. These expenses are often difficult to calculate because you have to assess how much more efficient your team will be with a better HPC solution and then work backwards to calculate the expenses correlated to inefficiency.
In summary, finding the true TCO of an on-premise HPC system can prove very difficult when considering all the hidden costs: staffing, facilities, power consumption, backup provisions, and forfeited rewards. I argue that one of the most important expenses to consider when comparing HPC systems is the expenses caused by forfeited rewards; however, these prove to be the most difficult to calculate and predict. The topic of TCO comparisons between cloud-enabled and on-premise HPC systems has been discussed regularly and is still not clearly defined. It is a comparison that we are working to improve, so if you have any comments or questions on this blog post or TCO, we would love to hear what you think.
Total Cost of Ownership (TCO) is a powerful financial tool that allows you to understand the direct and indirect expenses related to an asset, such as your HPC system. Calculating the TCO for an on-premise HPC system is direct: add up all expenses related to your system and its management for the entirety of its deployment. But what happens when you’re interested in switching to a cloud-enabled HPC environment? Can you confidently compare the cloud-enabled HPC system’s TCO with an on-premise HPC system’s TCO?
This question has been addressed by many different institutions.
Our view is simple: TCO is a poor financial tool for evaluating the value of cloud-enabled HPC. Comparing a system with a static environment against a dynamic environment creates an unreliable and misleading analysis. It is an apples to oranges comparison, and using TCO to assess cloud-enabled HPC attempts to make apple juice from oranges.
What is a static environment and how does it apply to my TCO analysis?
Static environments for TCO are used when you have set expense for a set return. For an on-premise system, you can get X amount of computing power for Y amount of dollars. This same relationship goes on for most expenses in the cost analysis of an on-premise HPC system until you reach a comprehensive TCO. There are some variable costs involved (fluctuation in software pricing, staffing, energy, unpredicted errors, etc.); however, margins can be used to monitor their influence on the TCO. Essentially, you end up with the general TCO analysis of X computing power = Y expenses ± margin of change. This is a great tool for comparing systems with little expense variations and known rewards that create a near-linear relationship. However, what happens when the computing power is nearly infinite, and the expenses are reactive, as is the case for cloud computing?
What is a dynamic environment and how does it apply to my TCO analysis?
A dynamic environment for a TCO analysis is a system where the expenses and rewards are not directly correlated, making them difficult to define and compare. In a cloud-enabled HPC system, you pay for computing power when you need it; there is little initial capital expenditure needed to use cloud-enabled HPC, when compared to on-premise HPC systems. In this environment, your expenses for HPC become less predictable and more reactive because they are generated from your computing demand. In addition, you are no longer constrained by a set capacity or architecture of computing resources, so your reward is extremely variable due to how you utilize HPC. This scalability and agility can heavily influence your HPC usage; especially if your current system is inhibiting your simulation-throughput and potential Design of Experiment (DOE). The rewards of cloud computing beckon the question: if you have less restrictions on HPC, would you utilize it differently?
What happens when you use TCO to compare on-premise vs cloud-enabled HPC systems?
TCO is a tool that is helpful for static environments, but when you try to take the same static tool and apply it to a highly dynamic environment, it is misleading. For example, consider you want to calculate the TCO of an on-premise HPC system. First, you must predict your peak usage and utilization for a system that will be used for approximately 3-5 years. To manage all an organization’s requirements, trade offs are made between peak capacity and the cost of obsolescence. Then you must pay the massive initial capital expenditure to purchase all the hardware, software, and staff required to assemble and operate the system. Calculate all these expenses and you receive your TCO for a system that awards you limited computing resources.
Now, try to use the same analysis of a cloud-enabled HPC system. Most take the projected peak computing power and average utilization and multiply it by the price to compute in their prospective cloud service provider. This is the first problem, you’re already treating both systems as if their rewards and expenses are equal. With cloud-enabled HPC systems, you have instant access to the latest hardware and applications which means you are always utilizing the best infrastructure for your workflow. By utilizing cloud resources, your computing power becomes near-infinite, meaning there is no reason to have a queue for running simulations, which increases your productivity. The limitless and diverse computing resources allows for innovations in the research and design process that are essential to getting better products to market before competitors. The inability to easily scale and upgrade resources for an on-premise HPC system can severely inhibit your ability to compete. The differences in rewards makes it hard to quantify the expenses associated with the aging on-premise HPC system’s effect on potential new workflows that can help you out-compete your competition.
When comparing HPC solution’s TCO, you must acknowledge the rewards provided by each solution, because the lack of a rewards should be reflected as an expense in the competitor’s TCO. For example, if your cloud computing solution provides no queue time, better computing performance, and new DOEs, but your on-premise solution doesn’t, then you must calculate the expenses of inefficiency correlated to the absence of rewards from the on-premise system. That is the only way to level the TCO with the corresponding rewards, but it proves extremely difficult to define exact numbers for each reward; henceforth, making TCO a misleading and inaccurate tool. Comparing the TCO and rewards of cloud-enabled and on-premise HPC systems is pointless because the tool does not address the reality of each system; one is static and requires massive investment to create limited computing power, and the other is agile and requires pay-as-you-go expenses for limitless computing power.
Determining the financial implications of incorporating cloud-enabled HPC into you HPC system can be difficult. Thankfully, Rescale has many specialists and confidential tools to help define the benefit of cloud-enabled HPC on your organization.
Accelerating innovation and powering big compute at hundreds of leading enterprises
With the year coming to a close, I’m excited to share the progress we’ve made at Rescale—not only as the leading Big Compute and cloud HPC solution in the market, but also as one of the fastest growing enterprise software companies of 2017. It’s been a busy year as we’ve landed over 100 new enterprise customers, fueling our rapid growth in multiple key industry verticals including aerospace, automotive, life sciences, universities and with broad geographic coverage throughout the Americas, Europe, and Asia.
Our customers are deeply engaging with the ScaleX Enterprise platform and our portfolio of industry solutions. Usage of Rescale has grown by 30% month-over-month throughout the entire year, resulting in numerous Silicon Valley investors identifying Rescale as the fastest growing enterprise software company in 2017. Customers are delivering incredible results accelerating their innovation capabilities, including: Airbus accelerating development of complex aerodynamics, LSIS deploying the first cloud-based engineering environment in Korea, RWDI implementing hybrid HPC across their enterprise, Boom Supersonic building the world’s next-generation supersonic jetliner, and many more.