pyslk.slk_retrieve(search_id, localpath) and pyslk.slk_retrieve(archpath, localpath) have different behavior
What happened
Expected behavior
import pyslk
pyslk.slk_retrieve("/arch/mh0010/m300408/showcase/dataset.zarr/.zmetadata", "/scratch/m/m300408/testretrieval/")
downloads the file .zmetadata
to /scratch/m/m300408/testretrieval/
. Because the preserve_path
option is by default False
, this is the expected behavior.
Unexpected/inconsistent behavior
However, in case I request the file not via the path but the search id, like so:
import pyslk
search_id = pyslk.slk_search_int('{"$and":[{"path":{"$gte":"/arch/mh0010/m300408/showcase/dataset.zarr","$max_depth":1}},{"resources.name":{"$regex":".zmetadata"}}]}')
pyslk.slk_retrieve(search_id, "/scratch/m/m300408/testretrieval/")
the file is downloaded to /scratch/m/m300408/testretrieval/arch/mh0010/m300408/showcase/dataset.zarr/.zmetadata
. This was unexpected because preserve_path
is still False
by default.
The latter seems to only occur since a recent update of pyslk/slk
. I observed this as it currently breaks slkspec
. I can easily adjust it, but want to make sure that the adjustment would last.
Versions
>>> pyslk.__version__
'1.6.2'
>>> module load slk
Autoloading openjdk/17.0.0_35-gcc-11.2.0
Autoloading slk_helpers/1.7.3
Loading slk/3.3.83
Loading requirement: openjdk/17.0.0_35-gcc-11.2.0 slk_helpers/1.7.3