| Port details |
- py-kerchunk Functions to make reference descriptions for ReferenceFileSystem
- 0.2.7 filesystems
 =0      Package not present on quarterly.This port was created during this quarter. It will be in the next quarterly branch but not the current one. - Maintainer: sunpoet@FreeBSD.org
- Port Added: 2025-05-29 04:54:18
- Last Update: 2025-05-29 04:52:18
- Commit Hash: 49cf560
- Also Listed In: python
- License: MIT
- WWW:
- https://fsspec.github.io/kerchunk/
- https://github.com/fsspec/kerchunk
- Description:
- Kerchunk is a library that provides a unified way to represent a variety of
chunked, compressed data formats (e.g. NetCDF, HDF5, GRIB), allowing efficient
access to the data from traditional file systems or cloud object storage. It
also provides a flexible way to create virtual datasets from multiple files. It
does this by extracting the byte ranges, compression information and other
information about the data and storing this metadata in a new, separate object.
This means that you can create a virtual aggregate dataset over potentially many
source files, for efficient, parallel and cloud-friendly in-situ access without
having to copy or translate the originals. It is a gateway to in-the-cloud
massive data processing while the data providers still insist on using legacy
formats for archival storage.
We provide the following things:
- completely serverless architecture
- metadata consolidation, so you can understand a many-file dataset (metadata
plus physical storage) in a single read
- read from all of the storage backends supported by fsspec, including object
storage (s3, gcs, abfs, alibaba), http, cloud user storage (dropbox, gdrive)
and network protocols (ftp, ssh, hdfs, smb...)
- loading of various file types (currently netcdf4/HDF, grib2, tiff, fits,
zarr), potentially heterogeneous within a single dataset, without a need to go
via the specific driver (e.g., no need for h5py)
- asynchronous concurrent fetch of many data chunks in one go, amortizing the
cost of latency
- parallel access with a library like zarr without any locks
- logical datasets viewing many (>~millions) data files, and direct
access/subselection to them via coordinate indexing across an arbitrary number
of dimensions
  ¦  ¦  ¦  ¦ 
- Manual pages:
- FreshPorts has no man page information for this port.
- pkg-plist: as obtained via:
make generate-plist - There is no configure plist information for this port.
- Dependency lines:
-
- ${PYTHON_PKGNAMEPREFIX}kerchunk>0:filesystems/py-kerchunk@${PY_FLAVOR}
- To install the port:
- cd /usr/ports/filesystems/py-kerchunk/ && make install clean
- To add the package, run one of these commands:
- pkg install filesystems/py-kerchunk
- pkg install py311-kerchunk
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above.
NOTE: This is a Python port. Instead of py311-kerchunk listed in the above command, you can pick from the names under the Packages section.- PKGNAME: py311-kerchunk
- Package flavors (<flavor>: <package>)
- distinfo:
- TIMESTAMP = 1748107898
SHA256 (kerchunk-0.2.7.tar.gz) = 0425aa0fbf56f898053ee4c4dd40b35cea12d2fc986e036086e99a4ad16bd4e6
SIZE (kerchunk-0.2.7.tar.gz) = 709052
No package information for this port in our database- Sometimes this happens. Not all ports have packages. This is doubly so for new ports, like this one.
- Dependencies
- NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
- Build dependencies:
-
- py311-setuptools>=42 : devel/py-setuptools@py311
- py311-setuptools-scm>=7 : devel/py-setuptools-scm@py311
- py311-wheel>=0 : devel/py-wheel@py311
- python3.11 : lang/python311
- py311-build>=0 : devel/py-build@py311
- py311-installer>=0 : devel/py-installer@py311
- Test dependencies:
-
- python3.11 : lang/python311
- Runtime dependencies:
-
- py311-fsspec>=0 : filesystems/py-fsspec@py311
- py311-numcodecs>=0 : misc/py-numcodecs@py311
- py311-numpy>=0,1 : math/py-numpy@py311
- py311-ujson>=0 : devel/py-ujson@py311
- py311-zarr>=0.1<3,1 : devel/py-zarr@py311
- python3.11 : lang/python311
- There are no ports dependent upon this port
Configuration Options:- No options to configure
- Options name:
- filesystems_py-kerchunk
- USES:
- python
- FreshPorts was unable to extract/find any pkg message
- Master Sites:
|
Number of commits found: 1
| Commit History - (may be incomplete: for full details, see links to repositories near top of page) |
| Commit | Credits | Log message |
|---|
0.2.7
29 May 2025 04:52:18
 |
Po-Chuan Hsieh (sunpoet) |
filesystems/py-kerchunk: Add py-kerchunk 0.2.7
Kerchunk is a library that provides a unified way to represent a variety of
chunked, compressed data formats (e.g. NetCDF, HDF5, GRIB), allowing efficient
access to the data from traditional file systems or cloud object storage. It
also provides a flexible way to create virtual datasets from multiple files. It
does this by extracting the byte ranges, compression information and other
information about the data and storing this metadata in a new, separate object.
This means that you can create a virtual aggregate dataset over potentially many
source files, for efficient, parallel and cloud-friendly in-situ access without
having to copy or translate the originals. It is a gateway to in-the-cloud
massive data processing while the data providers still insist on using legacy
formats for archival storage.
We provide the following things: (Only the first 15 lines of the commit message are shown above  ) |
Number of commits found: 1
|
As an Amazon Associate I earn from qualifying purchases.