Changes in pyslk v1.9.2#

updated: 2023-09-18 (removed note)

Summary for users who used releases of pyslk prior to 1.9.0#

A major structural change was implemented in pyslk version 1.9.0 compared to older versions. The implementation of this change did start in version 1.7.0 but most parts where implemented in 1.9.0. All functions which are meant to be used by end-users, can be used without specifying the module’s name: pyslk.example_function instead of pyslk.module_name.example_function.

The wrapper functions pyslk.parsers.*, which return handy datatypes and structures, were copied to pyslk.core.* and many new useful functions were added (API doc of functions with nice output<https://hsm-tools.gitlab-pages.dkrz.de/pyslk/api.html#wrapper-functions-which-return-nice-output>). These functions do not exactly match existing slk and slk_helpers commands but rather are oriented towards users needs and existing functions of other Python packages – e.g. pyslk.mkdir and pyslk.makedirs work very similar to os.mkdir and os.makedirs.

The simple wrapper functions pyslk.pyslk.slk_* / pyslk.slk_* are now available as pyslk.raw.*_raw / pyslk.*_raw (API doc of basic wrapper functions<https://hsm-tools.gitlab-pages.dkrz.de/pyslk/api.html#basic-wrappers-without-proper-parsing-of-the-output>). The original pyslk.pyslk.slk_* and pyslk.parsers.* functions are still available but are deprecated. We plan to remove all deprecated functions in pyslk 2.x.y.

Summary for users who never used pyslk before#

The Python library pyslk provides two types of wrapper functions:

  • basic wrapper functions which return the raw text output of the corresponding slk or slk_helpers command; each command has a corresponding wrapper; see API doc of basic wrapper functions<https://hsm-tools.gitlab-pages.dkrz.de/pyslk/api.html#basic-wrappers-without-proper-parsing-of-the-output>

  • advanced wrapper functions which return nice data types like boolean, integer, Pandas.DataFrame or whatever datatype/structure is reasonable in the context of the function’s usage; see API doc of functions with nice output<https://hsm-tools.gitlab-pages.dkrz.de/pyslk/api.html#wrapper-functions-which-return-nice-output>

These functions are located in different modules of the library. This does not matter for the user/you. All functions, which are meant to be used by end-users, can be used via pyslk.FUNCTION_NAME without providing the module’s name. The basic wrapper functions have the names [SLK_COMMAND_NAME]_raw and the advanced wrappers are mostly named independent of the slk / slk_helpers commands. Functions with the names like slk_* are old basic wrapper functions, which are deprecated. Please do not use them.

Detailed Changes#

  • pyslk adapted to slk_helpers version 1.9.7 and earlier releases

  • functions from the module pyslk.pyslk (== names have the prefix slk_) are marked as deprecated and have been replaced by *_raw functions (module pyslk.raw)

  • functions from the module pyslk.parsers are marked as deprecated and have been partly copied to the module pyslk.core; functions from pyslk.core than from pyslk.parsers although they have the same name

  • new: *_raw functions in module pyslk.raw (started with 1.7.0; mostly in 1.9.x):

    • the pyslk.raw module contains a wrapper function for each slk and slk_helpers command (same name + suffix _raw)

    • each *_raw function returns the raw text output of the respective slk / slk_helpers command

    • the *_raw functions raise errors if the slk / slk_helpers command returns non-zero exit codes

  • some functions in utils made public

    • convert_expiration_date (new in pyslk 1.9.0)

    • login_valid

    • run_slk

  • new: _nice_ wrapper functions in pyslk.code:

    • nearly all public functions which do not end on _raw, not start with slk_ and/or not throw deprecated warnings are from pyslk.core

    • some function names similar to those of the os library: mkdir, makedirs, chown (cannot change gid), chmod (does not follow definition of mode as in os.chmod)

    • intuitive function names (a selection): is_cached, is_on_tape, is_file, is_namespace, get_tag, get_metadata

    • handy return types:

      • pyslk.list(...) returns Pandas.DataFrame

      • pyslk.is_*, pyslk.has_* return booleans except for functions ending on _details

      • pyslk.*_details return dict with detailled file lists

      • dict are returned where reasonable (e.g. arch_size, job_queue, hsm2json, get_metadata)

Availability#

pyslk interacts with the StrongLink system via the command line interfaces slk and slk_helpers. Therefore, the slk/3.3.91 and slk_helpers/1.9.7 have to be loaded or your PATH needs to point to the bin folders of slk and slk_helpers. You cannot use pyslk outside of DKRZ to interact with StrongLink because access to StrongLink is only granted from within the DKRZ. However, you are welcome to install pyslk on your local machine in order to write code using pyslk outside DKRZ.

API documentation#

please see: https://hsm-tools.gitlab-pages.dkrz.de/pyslk/api.html

Changelog slk_helpers v1.9.5   HSM News for Oct 2023