Abstract:
Modern High Energy Physics (HEP) requires large-scale processing of extensive
amounts of scientific data. The needed computing resources are currently
provided statically by HEP specific computing centers. To increase the number
of available resources, for example to cover peak loads, the HEP computing development
team at KIT concentrates on the dynamic integration of additional
computing resources into the HEP infrastructure. Therefore, we developed ROCED,
a tool to dynamically request and integrate computing resources including
resources at HPC centers and commercial cloud providers. Since these resources
usually do not support HEP software natively, we rely on virtualization and container
technologies, which allows us to run HEP workflows on these so called
opportunistic resources. Additionally, we study the efficient processing of huge
amounts of data on a distributed infrastructure, where the data is usually stored
at HEP specific data centers and is accessed remotely over WAN. To optimize
the overall data throughput and to increase the CPU efficiency, we are currently
developing an automated caching system for frequently used data that is transparently
integrated into the distributed HEP computing infrastructure.
HPC