Wednesday, 28 June 2017

Is cloud inevitable for HPC?

In 2009, I wrote this article for HPC Wire: "2009-2019: A Look Back on a Decade of Supercomputing", pretending to look back on supercomputing between 2009 and 2019 from the perspective of beyond 2020.

The article opens with the idea that owning your own supercomputer was a thing of the past:
"As we turn the decade into the 2020s, we take a nostalgic look back at the last ten years of supercomputing. It’s amazing to think how much has changed in that time. Many of our older readers will recall how things were before the official Planetary Supercomputing Facilities at Shanghai, Oak Ridge and Saclay were established. Strange as it may seem now, each country — in fact, each university or company — had its own supercomputer!"
I got this bit wrong:
"And then the critical step — businesses and researchers finally understood that their competitive asset was the capabilities of their modelling software and user expertise — not the hardware itself. Successful businesses rushed to establish a lead over their competitors by investing in their modelling capability — especially robustness (getting trustable predictions/analysis), scalability (being able to process much larger datasets than before) and performance (driving down time to solutions)."
Hardware still matters - in some cases - as a means of gaining a competitive advantage in performance or cost [We help advise if that is true for our HPC consulting customers, and how to ensure the operational and strategic advantage is measured and optimized].

And, of course, my predicted rush to invest in software and people hasn't quite happened yet.

Towards the end, I predicted three major computing providers, from which most people got their HPC needs:
"We have now left the housing and daily care of the hardware to the specialists. The volume of public and private demand has set the scene for strong HPC provision into the future. We have the three official global providers to ensure consumer choice, with its competitive benefits, but few enough providers to underpin their business cases for the most capable possible HPC infrastructure."
Whilst my predictions were a little off in timing, some could be argued to have come true e.g., the rise to the top of Chinese supercomputing, the increasing likelihood of using someone else's supercomputer rather than buying your own (even if we still call it cloud), etc.

With the ongoing debate around cloud vs in-house HPC (where I am desperately trying to inject some impartial debate to balance the relentless and brash cloud marketing), re-visiting this article made an interesting trip down memory lane for me. I hope you might enjoy it too.

As I recently posted on LinkedIn:
"Cloud will never be the right solution for everyone/every use case. Cloud is rightly the default now for corporate IT, hosted applications, etc. But, this cloud-for-everything is unfortunately, wrongly, extrapolated to specialist computing (e.g.,  high performance computing, HPC), where cloud won't be the default for a long time.
For many HPC users, cloud is becoming a viable path to HPC, and very soon perhaps even the default option for many use cases. But, cloud is not yet, and probably never will be, the right solution for everyone. There will always be those who can legitimately justify a specialized capability (e.g., a dedicated HPC facility) rather than a commodity solution (i.e., cloud, even "HPC cloud"). The reasons for this might include better performance, specific operational constraints, lower TCO, etc. that only specialized facilities can deliver. 
The trick is to get an unbiased view for your specific situation, and you should be aware that most of the commentators on cloud are trying to sell cloud solutions or related services, so are not giving you impartial advice!"
[We provide that impartial advice on cloud, measuring performance, TCO, and related topics to our HPC consulting customers]


No comments: