OME-Zarr and napari
patchworks keeps everything in one OME-Zarr: the image as a pyramid, and each
segmentation under labels/<name>/ as a pyramid of its own with its
table. napari, Fiji/MoBIE and NGFF validators read it as is.
Convert
from patchworks.plugins.ome_zarr import to_ome_zarr
to_ome_zarr("scan.czi", "scan.zarr") # CZI, LIF, ND2, OME-TIFF, ... via bioio
to_ome_zarr("scan.ims", "scan.zarr") # Imaris
to_ome_zarr(array, "scan.zarr", axes="czyx", pixel_size={"z": 2.0, "y": 0.32, "x": 0.32})
Inputs are read lazily and the pyramid is written level by level, so any size
converts in bounded memory. The voxel size is read from the file and stored
in µm, and the channels' names too (napari names each channel's layer after
them; add them to an older store with patchworks fix-metadata scan.zarr
--channels-from scan.czi). Useful options:
| Option | Does |
|---|---|
n_levels=5 |
pyramid levels (X and Y halved each time; Z kept) |
axes= |
axis order of a bare array; from 4-D it is otherwise guessed (with a warning) |
reuse_pyramid=True |
copy an Imaris file's own pyramid instead of rebuilding it |
shard=True |
pack chunks into ~512 MB shard files: ~100× fewer files |
ngff_version="0.4" |
zarr v2 / NGFF 0.4 for older readers (default: 0.5 on zarr v3) |
compression= |
see Performance |
Install the readers you need: patchworks[bioio] (plus a native reader such
as bioio-czi for speed), patchworks[imaris].
A folder of single-plane TIFFs
One file per plane, such as sample_T0_Z000_C0_V0.tif: give a glob and a
regex whose named groups are the axes. A constant axis (T0) is dropped.
to_ome_zarr(
"sample/*.tif", "sample.zarr",
sequence_pattern=r"_T(?P<T>\d+)_Z(?P<Z>\d+)_C(?P<C>\d+)_V\d+",
shard=True,
)
On the cluster: sequence_pattern: in the config, with input: the glob.
The channels' names come from the files when they carry them (OME-XML,
ImageJ labels, Micro-Manager metadata) or from a companion *.ome.xml
beside them.
Labels in the store
tile_process("scan.zarr", fn) writes the labels into
scan.zarr/labels/labels (name it with output_component="cells"), with a
pyramid matching the image's. To add your own label array, or a pyramid to a
flat store:
from patchworks.plugins.ome_zarr import add_pyramid, write_labels
write_labels("scan.zarr", my_labels, name="nuclei")
add_pyramid("flat.zarr", base="0", n_levels=5)
A store written by an older version, or edited by hand, can be checked and
repaired with patchworks fix-metadata scan.zarr.
View in napari
from patchworks.plugins.napari import view_in_napari
view_in_napari("scan.zarr") # every channel and every label image
view_in_napari("scan.zarr", channel=0) # one channel
Only what is on screen is read, so it opens at once even for terabytes.
Each channel is a layer of its own, so several can be shown together; only
the first is visible at first (--show 0,2 for others), and a hidden
channel is not read until you switch it on. --slider puts all channels
on one layer's slider instead. From the cluster workflow:
pixi run -e viewer napari <work_dir>/image.zarr (needs a display). To
check and correct the results, see Reviewing.
If zooming or panning is slow, pixi run bench <image.zarr> (in
workflow/viewer) says where the time goes: the store's chunks, the
network, or drawing. A .zip on a network drive is read over the network
one chunk at a time: copy it to your computer and unzip it.
View in Neuroglancer, from the cluster
Neuroglancer runs in the browser and reads the store chunk by chunk, so nothing is copied: the store stays on the cluster and an SSH tunnel brings it to your browser. In a job (it reads the store as you browse):
It prints the tunnel to open on your computer (ssh -N -L ...) and a link
that opens Neuroglancer with each channel as a layer (named, coloured, the
first shown) and every label image as a segmentation layer: click an object
for its id, show only the ones selected. Every URL holds a random token, and
only the store is served, read-only. Serve the unzipped image.zarr, not
the bundle. No 3-D meshes: objects are shown in cross-section.
Neuroglancer fetches a whole chunk for any part of it, through the tunnel, so the server offers it every array in small chunks instead (one plane deep, at most 256 x 256), cut on request from the store's own and kept in memory a while (up to 2 GB, a third of the job's memory): a view costs the planes shown, not the 14 planes of a label chunk, and a side view one strip of each plane, not whole 1024 x 1024 planes. The store itself is not changed.
The viewer itself comes from the same server, through the same tunnel
(the neuroglancer package, in the workflow's environment; pip install
"patchworks[neuroglancer]" elsewhere): the page and the store share one
address. Without the package the link opens the public Neuroglancer, which
today's browsers stop from reading localhost ("blocked by CORS policy:
Permission was denied ... loopback address space" in the console) unless
the site is allowed Local network access (padlock left of the address,
Site settings).
Remote stores
Any fsspec URL works where a path does, for reading and for writing labels
back (pip install "patchworks[remote]"; credentials from the usual
AWS/GCP settings):
Scratch data stays on local disk, so only the final labels travel.