Abstract:
The operators of the federated research infrastructures at the involved HPC computer
centers face the challenge of how to provide storage services in an increasingly
diverse landscape. Large data sets are often created on one system and
computed or visualized on a different one. Therefore cooperation across institutional
boundaries becomes a significant factor in modern research. Traditional
HPC workflows assume certain preliminaries like POSIX file systems which cannot
be changed on a whim. A modern research data management aware storage
system needs to bridge from the existing landscape of network file systems into
a world of flexible scientific workflows and data management. In addition to the
integration of large scale object storage concepts, the long term identification of
data sets, their owners, and the definition of necessary meta data becomes a challenge.
No existing storage solution on the market meets all of the requirements,
and thus the bwHPC-S5 project must implement these features. The joint procurement
and later operation of the system will deepen the cooperation between
the involved computer centers and communities. The transition to this new system
will need to be organized together with the scientific communities being
shareholders in the storage system. Finally, the created storage infrastructures
have to fit well into the growing Research Data Repositories landscape.
Research Data Management