HPC community collaboration with real-life applications helps identify best
practices for next-gen systems
IO challenges in high performance data analytics. We need to
build tiered storage configurations so that we can take advantage
of scalability provided by technologies, such as object-based
storage, while enjoying the ease-of-portability and performance
of traditional POSIX-based parallel file systems. Therefore,
upcoming technologies, such as 2 TIERS and burst buffers, are
going to play a critical role in designing the IO subsystems for
next-generation HPC systems.
On the compute side, one of the goals for our HPC Innovation
Lab is to provide customers best practices on how to take the
most advantage of new processor and accelerator technologies.
To that end, we have been doing performance characterizations
with the new NVIDIA P100 GPUs using applications spanning
multiple domains (molecular dynamics, weather modelling,
manufacturing, etc.) and different system configurations. The
goal is to understand which applications benefit most from this
new technology and how to tune the systems in terms of their
PCIe topology or number of GPUs for optimal performance and
performance/watt under these workloads.
Another emerging trend for HPC systems is to use cloud
technologies to make HPC resources more flexible and easier to
manage. This trend manifests itself as offloading the appropriate
workloads to a public cloud, utilizing technologies such as OpenStack
or VMware to build private clouds for HPC, or a hybrid approach
where the public and private clouds are used in combination. We are
evaluating these technologies and conducting experiments to
understand their impact on system performance and ease-of-use.
Another important aspect of building complete HPC systems is the
software stack. To help the HPC community’s efforts in building an
Open Source and well-tested HPC software infrastructure, Dell EMC
became one of the founding members of the OpenHPC consortium.
In addition to work we have been doing with our software partners
(such as Bright Computing), we are also a contributor to the OpenHPC
project, both for testing software at scale and making contributions to
the development of the software itself.
Q: What current and potential future impacts are these
technologies having on the HPC industry?
Onur: The HPC industry spectrum spans from eager early
adopters of a new technology to risk-averse businesses, very large
enterprises to smaller ventures, and commercial entities to academic
and research-focused departments. New technology opens different
avenues for each of these different users. At the core, the goal is to
make HPC simpler, to allow users to spend more time on science
than on making the underlying technology work for them.
Developments in the areas of burst buffers and multi-tiered
storage architectures, such as 2 TIERS, will help organizations to take
HPC Source spoke to Onur Celebioglu, Engineering Director for the Dell EMC High Performance Computing solutions engineering team, about some of the cool new technologies
the Dell EMC HPC Innovation Lab is developing.
Q: What is the charter of the Dell EMC HPC
Onur: At Dell EMC, our HPC vision is to help more people in
industry, research, government and education use HPC solutions to
make more innovations and discoveries than any other HPC systems
vendor in the world. We plan to achieve this global impact by
building expertise to design, develop, deploy and support the most
capable, cost-effective portfolio of HPC solutions that integrate Dell
EMC innovations with community standards.
To aid us in achieving this goal, our HPC Innovation Lab is
dedicated to evaluating new technologies, doing performance
characterization with real-life applications and finding the best
practices to design and tune systems for specific workloads. The
HPC Innovation Lab staff consists of engineers with diverse areas
of expertise and a specialization in either an application domain or
a specific technology field. We also have world-class HPC systems
that allow us to do performance and scalability studies at scale.
Our goal is to collaborate with our technology partners, centers
of innovation and customers in finding innovative solutions to the
HPC community’s problems.
Q: Can you share with us some of the coolest new
technologies that Dell EMC is working on in the HPC
Onur: As you know, data-intensive computing has gained
wider adoption and has enabled many brand-new application
areas for HPC, especially in areas such as genomics, social
studies, etc. Availability of more sensors and the need
for finer granular models are driving growth in
data-intensive computing even for the more
traditional application areas of HPC. These trends
bring up the need for scalable storage technologies
that can deliver high performance, balanced with
cost and ease-of-use of the system. We are very
excited about the application areas for some of
the new technologies we have under the recently
combined Dell EMC product portfolio, such as
DSSD, ScaleIO, Isilon and ECS in the HPC space
that address these types of problems.
We are all aware that one single
storage technology cannot solve all