External Access

file version: 28 June 2021

External access to the HSM via sftp

From outside DKRZ data can be accessed read-only via sftp on your local computer:

sftp  <user>@xtape.dkrz.de

Before data can be copied from xtape.dkrz.de to another location, the data has to be located in the cache. slk list prints information whether a file is in cache or not. The permissions string has a length of 11 instead of 10 characters as we know it from ls (e.g. -rw-r--r--). The 11th character indicates whether the file is in the cache (-) or not in cache (t = tape). Below an example output of slk list is given. The files borehole_01.nc, nc3.nc and test.nc are available in the cache.

$ slk list /ex/am/ple/data | cat
-rw-r--r--- k204221     bm0146          1.2K   27 Mar 2020  borehole_01.nc
-rw-r--r--- k204221     bm0146          1.2K   04 Mar 2021  nc3.nc
-rw-r--r--t k204221     bm0146          1.2K   04 Mar 2021  nc_k_2.nc
-rw-r--r--t k204221     bm0146          4.0M   04 Mar 2021  nc_k_3.nc
-rw-r--r--t k204221     bm0146          4.0M   04 Mar 2021  nc_k_4.nc
-rw-r--r--t k204221     bm0146         13.1K   08 Dec 2020  small.nc
-rwxr-x---t k204221     bm0146        105.5M   08 Nov 2019  small_BPb4-Sl-mT_00062104_00040000000_01040000000.AGM07807972.freeze.nc
-rwxr-x---t k204221     bm0146        105.5M   14 Nov 2019  small_BPb4-Sl-mT_00062104_00040000000_01040000000.AGM07807972.nc
-rw-r--r--- k204221     bm0146          1.2K   23 Mar 2021  test.nc
-rw-r--r--t k204221     bm0146          1.2K   23 Mar 2021  zonk.nc
-rw-r--r--t k204221     bm0146          1.2K   23 Mar 2021  zonk2.nc
Files: 11

If a file is not located in the cache, you need to recalled it from tape into cache. This is done via slk recall. slk recall works like slk retrieve. But, instead of retrieving the data to somewhere, they are just loaded into the HSM cache. From there they can be copied via xtape.dkrz.de to an arbitrary location.

$ slk recall /ex/am/ple/data/zonk.nc

It might take some time until a file is recalled from tape depending on how many tape operations are currently performed by other users. If a file has been recalled to cache, the last character in tht permissions string changes from t to - (e.g. from -rw-r--r--t to -rw-r--r---).

$ slk list /ex/am/ple/data | cat
-rw-r--r--- k204221     bm0146          1.2K   27 Mar 2020  borehole_01.nc
-rw-r--r--- k204221     bm0146          1.2K   04 Mar 2021  nc3.nc
-rw-r--r--t k204221     bm0146          1.2K   04 Mar 2021  nc_k_2.nc
-rw-r--r--t k204221     bm0146          4.0M   04 Mar 2021  nc_k_3.nc
-rw-r--r--t k204221     bm0146          4.0M   04 Mar 2021  nc_k_4.nc
-rw-r--r--t k204221     bm0146         13.1K   08 Dec 2020  small.nc
-rwxr-x---t k204221     bm0146        105.5M   08 Nov 2019  small_BPb4-Sl-mT_00062104_00040000000_01040000000.AGM07807972.freeze.nc
-rwxr-x---t k204221     bm0146        105.5M   14 Nov 2019  small_BPb4-Sl-mT_00062104_00040000000_01040000000.AGM07807972.nc
-rw-r--r--- k204221     bm0146          1.2K   23 Mar 2021  test.nc
-rw-r--r--- k204221     bm0146          1.2K   23 Mar 2021  zonk.nc
-rw-r--r--t k204221     bm0146          1.2K   23 Mar 2021  zonk2.nc
Files: 11

After successful recalling, the file(s) can be copied via sftp (see first code block). The directory structure is the same as when data is accessed via slk.