SMuK 2023 – wissenschaftliches Programm
Bereiche | Tage | Auswahl | Suche | Aktualisierungen | Downloads | Hilfe
T: Fachverband Teilchenphysik
T 62: DAQ NN/ML – GRID II
T 62.6: Vortrag
Mittwoch, 22. März 2023, 17:05–17:20, HSZ/0301
Sapphire - Small-file aggregation for the dCache tape interface — •Svenja Meyer1, Krishnaveni Chitrapu3, Dmitry Litvintsev2, Paul Millar1, Tigran Mkrtchyan1, Lea Morschel1, Albert Rossi2, and Marina Sahakyan1 — 1Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany — 2Fermi National Accelerator Laboratory, Batavia, USA — 3National Supercomputer Center, Linköping University, Sweden
dCache is an open source distributed storage system used to manage and store scientific data in the scale of hundreds of petabyte. Archiving data on tertiary storage, for example tape, is a main feature of this software. Unfortunately the performance of writing data to tape decreases for small sized files, which are produced more and more by experiments.
To circumvent this problem, Sapphire, an advancement of SmallFiles for dCache, was developed. Working as a plugin for dCache, these small files are bundled into bigger archives without needed intervention by the user. Flushing files to tape as well as staging them back works transparently to the user.