Navigating the Cloud HPC Solutions for Enterprise IT


The above tweet in my newsfeed caught my attention because it succinctly echoed thoughts several IT leaders have recently shared with me. Recent research, like this study from Accenture, reinforce Tim’s observation:

  • 95% of respondents have a five-year cloud strategy already in place
  • Four of five executives reported that less than half of their business functions are currently operated in public cloud, but noted increasing intent on moving more of their operations to the cloud in the coming years
  • 89% of respondents agree that implementing cloud strategies is a competitive advantage which allows their companies to leverage innovation through agility
  • While half of respondents cite security as their biggest concern with the public model, more than 80% believe public cloud security is more robust and transparent than what they’re able to provide in-house

Frustrating the “how” decisions by IT leaders is the velocity in the expansion of the cloud market. The low barrier to entry is flooding the market with SaaS, IaaS, and PaaS technologies but the overwhelming amount of options are as likely to lead to paralysis as decision. Selecting an enterprise high-performance computing partner with the right strategy – the “how” – is critical. If you want more thoughts on the “why,” a link at the end of the article will share a perspective from one of our partners.

Here are five characteristics IT leaders should look for when selecting a cloud solution for high-performance computing:

The service has an enterprise strategy. This is seemingly obvious, but is a question worth asking when surveying a variety of available solutions. A service that has an enterprise strategy is a service that supports a diverse customer environment (variety of software vendors, software tools, workflows and/or hardware types) and replaces the burden of IT administration with IT management control. In our experience, enterprises commonly need:

1) Scalability
2) Flexibility
3) Compatibility with hybrid environments
4) Support for a diversity of workflows
5) Management tools
6) Integration strategies

Scalability, flexibility, and hybrid environments will be addressed below. The other three elements (workflows, management tools, and integration) are important, but largely a feature and function discussion that we’ll take on in another article.

The service is scalable for the enterprise. Product design, data research, and R&D cycles — and thus their corresponding infrastructure needs — are anything but smooth or predictable. Therefore, an enterprise solution needs to be able to rapidly scale from sixteen to thousands of cores to meet the needs of the enterprise. The engineering team that must undertake a massive redesign in the 11th hour should not be capped by limited capacity and queues. Delivering a solution to this problem is simply expensive without a service that offers wide-ranging access to multiple public clouds with on-demand pricing options.

The solution supports hybrid environments. As the Accenture study showed, many organizations are taking a “cloud first” approach. However, the diversity of workloads in the enterprise environment, varying size of data sets, and legacy systems and operations will likely keep on-premise infrastructure around for the short-term. CIOs may be “getting out of the data center business,” but as companies move along the cloud spectrum, there will be some companies that have a need for a mixed model. Thus, an enterprise solution allows organizations to manage both cloud and on-premise resources from a single platform. Not only does this assist the IT team with management, monitoring, and control, but also simplifies the experience for the end user.

The solution features the best of the cloud’s flexibility and agility. Much to the chagrin of IT leaders, hardware requirements for engineering, data scientists, and researchers are becoming increasingly more diverse. As hardware is increasingly optimized for different purposes and software developers are optimizing codes on platforms with divergent strategies, flexibility should be a concern of any IT leader. Enterprises need the agility to support the demands of an enterprise software tool suite – both today and tomorrow. One-size-fits-all solutions for HPC hardware will become obsolete in an enterprise with a sizable engineering staff (or even a small, but diverse staff). Enterprise flexibility and agility is delivered by a single platform/environment that brings different clouds, hardware, software, and pricing models together in one place. Alignment of cost and demand is the promise of the cloud. With a wide variety of hardware available, enterprises should be able to make decisions about computing that align with business drivers. What does this mean in practice? This means being able to take advantage of everything from low-cost public cloud spot markets to instantly-available, cutting-edge processors. This also means not requiring lock-in to pre-paid models that expire. Select a partner that can navigate the cloud environment on behalf of the enterprise’s evolving requirements.

The solution is secure. I included security because without it, invariably a reader would find this list woefully incomplete. However, I would contend that a vast majority of mature cloud HPC solutions can satisfy the requirements of 98% of companies. More often, security concerns are a red herring to disguise a cultural or organizational bias against a “XaaS” approach.

What about data transfer and large data sets? I often get this question, for good reason. There are several mitigation strategies for this issue. However, an entire article should be dedicated to this topic – we’ll follow-up on that in a later post.

To conclude, as the market for cloud services rapidly expands and most organizations are satisfied with the answers to “why” cloud, enterprise IT is faced with a multitude of decisions on the “how” cloud. For high-performance computing, this article should give managers some critical factors to consider when evaluating solutions.

And one last thing, a “why” cloud link, as promised: 6 Advantages of Cloud Computing

This article was written by Matt McKee.