Skip to content

Changelog

0.10.2 (2026-11-13)

0.10.1's fix, on tensorstore 0.1.86. No placement change, no unit table change, and no default changes.

  • A gs:// open with a stale Google login falls back to the public https URL on tensorstore 0.1.86 too. 0.10.1 knew the failed login by the source locations in tensorstore's error, which 0.1.86 leaves out, so with the newest tensorstore (what a fresh install gets) the open still raised. The token endpoint's own error code (invalid_grant, and the other OAuth 2.0 codes) is now matched as well. On 0.1.86 a token endpoint out of reach is raised as any unreachable host is, not passed over.
  • CI also runs the tests on the newest release of every dependency, weekly and on every pull request; 0.10.1 passed on the lock's tensorstore 0.1.78.

0.10.1 (2026-11-13)

A fix for gs:// with a stale Google login. No placement change, no unit table change, and no default changes.

  • A gs:// open falls back to the bucket's public https URL when Google's default credentials fail to refresh. An expired gcloud login (a Workspace reauth policy) is refused with HTTP 400 invalid_grant, which tensorstore does not report as a refusal, so the open raised instead of reading a public bucket. Any error from getting the access token now counts as a refusal, and so does a token endpoint out of reach.
  • When every way in fails after a refusal, the error raised carries the first refusal as its cause, so a stale login stays in the traceback of a private bucket's 403.

0.10.0 (2026-11-03)

Axes in another order, on request. No placement change, no unit table change, and no default changes: without axes_order, every array opens as it did in 0.9.

  • open, open_pyramid and metadata take axes_order: "stored" (the default, for good), "zyx" (the axes named z, y and x last, in that order, the others first in stored order) or a sequence naming each axis once. An OME-Zarr written x, y, z (TensorSwitch's) then reads as z, y, x, and a channel-last array channel first. The names move with the data, and shape, chunks, shards, units, voxel size and translation with them, in .ts, meta, grid, reads, writes, blocks, a pyramid's placements and level_meta, level choice by voxel size, base_voxel_size, reopen and a pickle. open_tensorstore stays in the store's order.
  • On such a handle, chunk_key, parse_chunk_key and list_chunks take and give indices in the presented order (a selected axis in its stored place); the keys are the store's. add_level takes its per-axis arguments in the pyramid's order and creates the array in the group's.

0.9.1 (2026-10-26)

A fix for exit. No placement change, no unit table change, and no default changes.

  • The watchdog thread stops at exit, before the interpreter finalizes. On Python up to 3.13 a process could abort once all its work was done ("terminate called without an active exception", exit 134) when finalization began while the thread was inside a tensorstore call; an LSF job that did this looked failed. Now an atexit handler, which runs once the program's other threads are joined, ends the thread and waits for it, and no new one starts after that.

0.9.0 (2026-10-23)

A job script can size each worker's threads. No placement change, no unit table change, and no default changes: with WORLDARRAY_THREADS unset and no file_io_floor, every context is what 0.8 made.

  • thread_budget() reads WORLDARRAY_THREADS ahead of OMP_NUM_THREADS and the job's slots, for a job script to give each dask worker its share of the job's cores where OMP_NUM_THREADS stays 1 for numpy and LSB_DJOB_NUMPROC counts the whole job (for example max(1, ncpus // processes)). A handle opened on the driver on the shared context is sized in each worker by that worker's value. Anything but a positive integer raises ValueError; blank is unset.
  • shared_context(file_io_floor=) sets the least the file I/O pool takes under the thread budget (default FILE_IO_FLOOR, 8); 1 keeps file I/O at the budget. A pickle carries it, and with threads=N or threads=0 it is ignored.

0.8.0 (2026-10-23)

Faster writes from the contexts the package makes, and the create open timed out. No placement change and no unit table change. Requires tensorstore 0.1.68 or later (was 0.1.60), the first with the file_io_locking setting.

  • shared_context and capped_context write without fsync (file_io_sync: false) and without OS file locks (file_io_locking mode "none"); fsync=True and locking="os" restore tensorstore's defaults. A whole-unit write stays atomic, a temporary file renamed into place; only concurrent read-modify-writes of one unit (partial=True) lose their guarantee, which was never given (Behaviour 13). fsync cost 6.5 ms a chunk on a local disk and nothing on NFS; the locks cost about 10 ms a chunk on NFS.
  • The default context's file I/O pool is thread_budget() or 8, whichever is larger; decoding stays at thread_budget(). File I/O threads wait on storage rather than compute, and one of them wrote a block's chunk files one NFS round trip after another. capped_context(file_io=) sizes it apart from concurrency (default unchanged).
  • From one-thread workers through the default context, against 0.7: a 64^3 label chunk to NFS 23 to 12.2 ms (zarr-python 2: 13.1), an 8-chunk label block 159 to 66.5 ms (zarr-python 2: 102), an 8-chunk EM block 330 to 202 ms; to a local disk a label block 51.5 to 4.1 ms. Through capped_context(1, network=False) the NFS label block went 166 to 91 ms; file_io=8 gives it the rest.
  • The open that creates an array is timed out and retried under retries (Behaviour 6). A retry that finds the array made by an attempt of its own that timed out adopts it after the usual comparison, even under exists="error". That open keeps OS locking, so create-if-absent stays atomic.

0.7.1 (2026-10-23)

Faster opens over http. No placement change, no unit table change, no API change.

  • The documents a call needs from one directory are read at once instead of in turn. An http open of a zarr v2 level made its eight requests one after another (five probe keys, the level's .zattrs, the parent's zarr.json and .zattrs); now they go in three rounds. At 10 ms an answer, wa.open took 98 ms and takes 52 (cellmap-flow's measurement). A server that never answers still costs one key's attempts, not five.

0.7.0 (2026-10-23)

Opens time out. No placement change and no unit table change.

  • Every metadata document read or written and every store opened is timed out and retried under Retry, as data reads are (Behaviour 6): by default 5 s times the attempt, ten attempts. A server that accepts a connection and never answers held wa.open and wa.metadata forever before; it now raises TimeoutError naming the URL and the attempts (cellmap-flow, 2026-10-23). The policy is the call's retries, which wa.metadata, wa.detect and wa.open_tensorstore now take as wa.open, wa.open_pyramid and wa.create already did. Listings and the open that creates an array are not timed out.

0.6.0 (2026-10-23)

One new argument, at chunkmirage's request. No placement change and no unit table change; nothing changes for a caller that does not pass it.

  • wa.shared_context(cache_bytes, threads=): None (the default) limits the decoding and file I/O pools to thread_budget() as in 0.5, a positive number limits them to that many, and 0 writes no limit, so tensorstore sizes them itself (every CPU on the node). Each (cache_bytes, threads) is one watched context for the process, so a server can share one cache pool across every store it opens at the thread count it chose, without building and watching its own context. A pickled handle on such a context comes back on the same one; threads=None still lets the other process size by its own job.
  • Documented what a pool of one thread costs (spec Behaviour 7, capped_context_spec): nothing on one-chunk blocks; on 8-chunk blocks, half the speed of 4 in a two-slot job but only 7 to 19% slower in a one-slot one (c-elegans on /nrs), so a worker per slot loses little at the default.

0.5.0 (2026-10-23)

Two bugs that hung or crashed cellmap-analyze 0.6 on /nrs (salamander's and c-elegans' runs, 2026-10-22), and tensorstore's threads sized to the job. No placement change and no unit table change.

  • A read whose first attempt timed out raised IndexError instead of retrying: the warning took the first line of the error's message, and tensorstore's TimeoutError() has none. The warning now names the error's type when the message is empty, and the error after the last attempt names the operation, the path and the attempts.
  • Every context the package makes or opens a store on is watched: once a second a daemon thread gives its decoding and file I/O pools one no-op task each (an in-memory copy and a local read of a missing key; no store is touched). tensorstore up to at least 0.1.85 can strand a queued task: a pool worker idle for 20 ms exits as the task arrives, and the task never runs (fixed upstream in google/tensorstore ed51718085, in no release yet). With data_copy and file_io capped at 1, as cellmap-analyze's workers were, dozens of workers per run waited forever in write_index with no tensorstore thread alive, and reads timed out into the IndexError above. One more task on the same pool wakes it, which salamander's and c-elegans' sessions proved on live stuck workers, so a stranded task now runs within about a second, at any concurrency, whoever waits on it: worldarray, vol.ts directly or await. Reproduced on purpose on tensorstore 0.1.75, 0.1.78 and 0.1.85 (opt-in tests/test_lost_wakeup.py): cellmap-analyze's block loop gave 29 IndexErrors and 55 writes stuck past 20 s on 0.4.0, none on 0.5.0, slowest 0.98 s. Spec open question 14 says when this goes.
  • write_index, write and write_chunk are timed out and retried as reads are (Behaviour 6), by a new retries= argument or the handle's retries. A write used to wait on its future with no timeout at all.
  • Without context, the shared context's decoding and file I/O pools are limited to the new wa.thread_budget(): OMP_NUM_THREADS when set (dask sets 1 in each worker), else the job's slots (LSB_DJOB_NUMPROC, SLURM_CPUS_PER_TASK, NSLOTS), else the CPUs the process may run on. A server that wants more threads than its job's slots asks for more slots or passes its own context. tensorstore alone sizes both to every CPU on the node, whatever bsub -n asked for, so a dask worker's tensorstore now takes the same one thread as its numpy (pinned to one CPU, tensorstore's default context started 43 to 45 pool threads, the capped one 4 to 5). Against cellmap-analyze 0.5.0's own block I/O on NFS (128^3 uint64 label blocks, each one chunk), its 0.6.1 on this release reads a block in 26 ms instead of 40 and writes it in 45 ms instead of 62 from one process at one thread per pool, and about 1.15 times faster from 8. Speed gains grow with blocks of several chunks and more threads.

0.4.0 (2026-10-22)

cellmap-analyze's three asks, so it can call worldarray directly with no wrapper, and two of cellmap-flow's. No placement change and no unit table change.

  • vol.read_onto(grid, begin=None, end=None, ...) reads voxels [begin, end) of a Grid's spatial axes from the array, exactly that shape (Behaviour 20). It is the read of the grid's index box at the grid's voxel size anchored at its corner, so at the array's own voxel size nothing is resampled, whatever the sub-voxel offset, and otherwise the array is resampled onto the grid. The grid's units are converted.
  • A pickled handle, and reopen, use the path resolved against the working directory at open, so a worker that changes directory before unpickling opens the same store; it used to raise FileNotFoundError for a relative path.
  • write_index, write and write_chunk take a scalar, which fills the box. It is converted to the dtype only when its value survives (0 into uint8; 300, -1, 0.5 and NaN raise), and the fill value over whole units leaves them absent.
  • vol.source_translation: the stored level's translation in units, beside source_voxel_size, under resample and relabel too (cellmap-flow's stored_placement).
  • create(exists="adopt") also compares the keys of attrs with the existing array's attributes and raises naming each that differs or is missing; keys not given are not compared. An adopt given different attrs used to keep the old ones silently, as funlib's prepare_ds kept an old offset (cellmap-flow's _refuse_another_region).

0.3.1 (2026-10-22)

  • source_voxel_size reports the stored level's voxel size under on_mismatch="relabel", as the spec says; it reported the relabelled size (cellmap-flow's report). Reads, voxel_size, translation and corner are unchanged.
  • box of an array opened with a resample target is the extent of the target voxels shape counts, as the spec's [corner, corner + shape * voxel_size) says and as Grid.from_meta(meta, voxel_size=t).box already was; it was the stored data's extent, so read(box) could return fewer voxels than shape (cellmap-flow's 7 voxels of 8 nm at 24 nm: box ended at 56 nm, now 72). No voxel moves.
  • Unit table change. The unit table knows kilometres (kilometer, kilometre, km, 1e3 m), so they convert, abbreviate to km and are written as OME's kilometer (chunkmirage's ask). This should have been a minor release (spec section 8): cellmap-flow exports the table and its CI failed until it moved its pin to >=0.3.1.
  • README: the placement rule gains the corner formula, a worked example, why the centre is the model, and how the read rule follows from it.

0.3.0 (2026-10-22)

Grids and blocks, built to spec draft 3 (Behaviours 17 to 19). Nothing on the read or write side changes.

  • blocks(shape, block_shape, context=0, box=None, clip=True): the array (or a voxel box, which may reach past the array) cut into write boxes that tile it with no overlap and no gap, each with a read box grown by context (one int, one per axis, or (before, after) pairs) and clipped to the array when clip; C order, len, blocks[id] in O(1), index_of(begin); a Block carries index, id, begin, end, read_begin, read_end, at_start and at_end per axis and inner, the write box's slices inside the read box. Plain picklable records; nothing is scheduled. vol.blocks() defaults the block to write_shape, so default blocks are aligned writes. vol.write_chunk(index, data) writes one whole unit.
  • Grid(shape, voxel_size, translation, axes, units, chunks, shards): a WorldArray without a store, with box, corner, index_box, chunks_covering, write_shape, blocks and meta(dtype); Grid.from_meta(meta, voxel_size=) is an existing array's grid or the resample target grid over it. create(path, grid=g, dtype=) creates from it.
  • level_factors(voxel_size, ratio=1.5, ...): the anisotropy-aware pyramid schedule mesh-n-bone and cellmap-analyze share, per level.
  • units.abbreviation, and store.join, store.parent, store.basename as public URL-aware path helpers.

0.2.0 (2026-10-22)

Metadata and placed creation, built to spec draft 3 (Behaviours 10 to 16). Nothing on the read side changes.

  • ome_multiscales(axes, units, levels, version) builds the complete OME-NGFF 0.4 or 0.5 group document with one explicit placement per level; level_placement and level_shape derive a level from factors and an origin (Convention 1's rule inverted, so a level built from a ROI origin is placed from it; level_shape absorbs float-noise factors such as 0.3 / 0.1). Units are written in OME's names ("nm" becomes "nanometer"; units.ome_name), an unknown unit as given; the multiscale's type is written when given. Expected documents are pinned in tests/fixtures/write.json, shared with chunkmirage's TypeScript builder, each case with its resolved per-axis units and types, fifteen cases.
  • create(path, shape, dtype, voxel_size, translation, chunks=..., ...) writes the group document (made when absent, else its one dataset entry added or replaced, every other attribute kept) and then the array, as zarr v2 with OME 0.4 or zarr v3 with OME 0.5 following the group's format. chunks is required, codecs are explicit with a zstd level 3 default, shards are a zarr v3 option. exists is "error" or "adopt" (an identical array is opened for writing, a different one raises naming every difference, a creation lost to another process is adopted after a short backoff). Nothing is ever deleted, truncated or overwritten. Write units equal to the fill value are not stored unless store_fill=True, and a stored unit written so loses its key, as zarr-python's write_empty_chunks=False does. create also writes a bare group document for every missing ancestor down from the .zarr root and follows the nearest existing ancestor's zarr version, so a level made in a zarr v2 container is zarr v2 with OME 0.4 and the hierarchy browses (cellmap-analyze, cellmap-flow). Adopting an identical array writes no group document; after a write the document is read again and a lost entry is written again, then raises (registration-station's concurrent block jobs).
  • ensure_group_entry is public: list an array the package did not create, such as a symlinked s0 (mesh-n-bone), or a level filled before it is listed.
  • open(path, mode="r+") opens an existing array for writing; a WorldArray pickles by its arguments and its context's spec and reopens in the receiving process, a capped context staying capped (mesh-n-bone's workers). write.codecs_of takes a path.
  • write_index(begin, end, data) writes whole write units only (the shard when sharded, else the chunk; the clipped last unit allowed), so writers in different processes need no coordination; partial=True does tensorstore's read-modify-write. write(box, data) maps a world box by the centre rule first. No cast, no padding, no resampling on write.
  • open_pyramid(path, mode="r+").add_level(name, factors=, origin=) creates a level's entry and empty array, placed and sized from the level below with its dtype, chunks, shards, codecs and fill value copied; the downsampling itself stays with the caller.
  • chunk_key and parse_chunk_key for zarr v2, zarr v3 (both encodings, sharded arrays keyed by shard), N5 and precomputed, as free functions and on WorldArray; list_chunks uses the same parser. write_shape, ArrayMeta.shard_shape, chunks_covering(write=True).

0.1.1 (2026-10-22)

  • funlib voxel_size/offset on the parent group place an array that carries none, for zarr and N5 alike; funlib's keys on an N5 array are read in the array's C order, as zarr-python's N5 store writes them (both from mesh-n-bone's review of its PR #48).
  • A precomputed path that names a scale is that scale under level=0 or "first" too; other levels warn and are ignored, as on an array path.
  • The GCE metadata guard is applied only off Google Cloud (decided from the DMI product name and the GCE_METADATA_* variables), so metadata-server credentials keep working on a Google VM.
  • Pyramid.placements(), voxel_sizes() and translations() read the group document when the pyramid was opened through one of its level paths too; they used to open every level from there, and a missing level made them raise.
  • Placement change. An N5 level with downsamplingFactors and no offset now sits with its corner at the origin at every level, as Neuroglancer draws N5: translation f / 2 * base. 0.1.0 followed BigDataViewer for pixelResolution pyramids, centring the base's voxel 0 on the origin ((f - 1) / 2 * base, half a base voxel lower), while placing resolution pyramids with the corner at the origin, so the two disagreed. Two shared fixture cases move and are renamed: n5_group_pixelresolution_and_downsampling_factors_corner_at_origin (translation 4, 4, 4) and n5_bigstitcher_resolution_with_downsampling_factors_corner_at_origin (0.5, 0.157, 0.157).

0.1.0 (2026-10-21)

  • First version, built to docs/spec.md: open zarr v2, zarr v3, N5 and Neuroglancer precomputed arrays and pyramids through tensorstore; one metadata precedence per format with the voxel-centre rule for translations; index-space and world-box reads with padding; level selection; nearest and linear resampling; retries; shared and capped tensorstore contexts. Built and trialled in one day against the readers it replaces in cellmap-flow, cellmap-analyze, mesh-n-bone and chunkmirage, and checked on their own data by the CineMap, liconn-err, registration-station and salamander repos; see docs/spec.md, "Trial status". Hot-path reads cost 0.1 to 0.2 ms over a raw tensorstore read.