Getting Started with slk¶
file version: 15 October 2021
current slk version: 3.3.6
slk comands behave very similar to known commands on the Linux terminal (POSIX compliant):
However, there are a few differences which might cause issues if users expect a fully POSIX-compliant slk. Please read slk pitfalls for details. What we know as a
directory is called
namespace in StrongLink.
Before you can use the slk commands the first time, you must call slk login and login with your DKRZ/mistral/luv login credentials.
Load slk module¶
slk is available as module on all mistral nodes. Just do
$ module load slk
to load the most recent slk version. If you wish to use a specific old release (e.g. slk version 3.1.47), please do
$ module load slk/3.1.47
On which nodes to run slk¶
We suggest running
slk archive and
slk retrieve on the
compute2 nodes. The run time on the
mistralpp nodes considerably depends on the activity of other users on these nodes.
Please do not run
slk archive and
retrieve on the mistral login nodes (
mlogin10X) when you archive large amounts of data because
slk causes high CPU load and uses much memory.
The available memory per job on the
shared nodes is very low. Therefore,
slk archive and
slk retrieve are slower than on other nodes. The run time can be expected to be two to four times as long as on the
slk login and use your common DKRZ credentials to log in. You do not login to another shell with
slk login (like
pftp) but it creates a login token. This login token allows the usage of the other slk commands. It is located in
~/.slk/config.json and is valid for 30 days.
$ slk login
All mistral nodes share the same home directory. Therefore, you need to login only once and you can run it on any mistral node.
Archive a file or directory¶
slk archive can be used to archive files or directories. A progress bar is printed when
slk archive is used in interactive mode (session on
mistralpp or in an interactive job via
slk archive is used in a batch script, no output will be written into the job log. An example
slk archive call would be:
$ slk archive /work/bm0146/k204221/some_file.nc /arch/bm0146/k204221/my_data/
slk archive allows the usage of
* as wildcard (but not
-R to do recursive archival of directories.
slk archive works semi-recursive without
-R (see slk pitfalls for details).
List content of a namespace¶
slk list automatically prints its findings in a pagination mode with 25 items per page. That means that you see the first 25 results and have to type
Enter to show the next 25 results. The pagination mode is deactivated when the output of
slk list is piped (
|) into another command – e.g.
$ slk list /arch/bm0146/k204221/my_data | cat -rw-rw-rw-- k204221 bm0146 1.2K 27 Mar 2020 borehole_01.nc -rw-rw-rw-- k204221 bm0146 1.2K 04 Mar 2021 nc3.nc -rw-rw-rw-- k204221 bm0146 1.2K 04 Mar 2021 nc_k_2.nc -rw-rw-rw-- k204221 bm0146 4.0M 04 Mar 2021 nc_k_3.nc -rw-rw-rw-- k204221 bm0146 4.0M 04 Mar 2021 nc_k_4.nc -rw-rw-rw-- k204221 bm0146 13.1K 08 Dec 2020 small.nc -rwxrwx-w-- k204221 bm0146 105.5M 08 Nov 2019 small_BPb4-Sl-mT_00062104_00040000000_01040000000.AGM07807972.freeze.nc -rwxrwx-w-- k204221 bm0146 105.5M 14 Nov 2019 small_BPb4-Sl-mT_00062104_00040000000_01040000000.AGM07807972.nc -rw-rw-rw-- k204221 bm0146 1.2K 23 Mar 2021 test.nc -rw-rw-rw-- k204221 bm0146 1.2K 23 Mar 2021 zonk.nc -rw-rw-rw-- k204221 bm0146 1.2K 28 Jun 2021 some_file.nc Files: 11
Retrieve a file or namespace¶
slk retrieve is the counterpart of
slk archive. It behaves quite the same as
slk archive (progress bar;
-R) but the * works only for files and not for namesspaces.
slk retrieve /arch/bm0146/k204221/my_data/nc3.nc /work/bm0146/k204221/results
Move, rename and delete files or namespaces¶
Move a file from one namespace to another. The file’s name cannot be changed by this command.
$ slk move /arch/bm0146/k204221/my_data/nc_k_2.nc /arch/bm0146/k204221/old_data
Rename a file. The file’s location cannot be changed by this command.
$ slk rename /arch/bm0146/k204221/my_data/nc_k_3.nc a_netcdf_file.nc
Delete a file. If we apply
slk delete onto a namespace then it deletes all files in this namespace without confirmation. Files in sub-namespaces are not deleted. To delete the whole namespace with all sub-, subsub-, …-namespaces and their content, please append
$ slk delete /arch/bm0146/k204221/my_data/zonk.nc