Skip to content

Latest commit

 

History

History
330 lines (213 loc) · 11.5 KB

CHANGELOG.rst

File metadata and controls

330 lines (213 loc) · 11.5 KB
Raw

Wendelin.core change history

  • 2.0.alpha4 (2024-09-19) More fixes discovered by on-field usage, added new ZBlk format auto and implemented protection against faulty clients.
  • 2.0.alpha3 (2022-12-21) More fixes discovered by on-field usage.
  • 2.0.alpha2 (2022-01-27) Fix several crashes discovered by first on-field usage.

2.0.alpha1 (2021-11-16)

This is a major pre-release that reduces wendelin.core RAM consumption dramatically:

The project switches to be mainly using kernel virtual memory manager. Bigfiles are now primarily accessed with plain OS-level mmap to files from synthetic WCFS filesystem. This makes bigfile's cache (now it is the kernel's pagecache) to be shared in between several processes.

In addition a custom coherency protocol is provided, which allows clients, that want to receive isolation guarantee ("I" from ACID), to still use the shared cache and at the same time get bigfile view isolated from other's changes.

By default wendelin.core python client continues to provide full ACID semantics as before.

In addition to being significantly more efficient, WCFS also fixes data-corruption bugs that were discovered in how Wendelin.core 1 handles invalidations on BTree topology change.

Please see wcfs.go for description of the new filesystem.

Major steps: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21.

0.13 (2019-06-18)

  • Add support for Python 3.7 (commit).
  • Add RAMArray class which is compatible to ZBigArray in API and semantic, but stores its data in RAM only (commit 1, 2).
  • Add lib.xnumpy.structured - utility to create structured view of an array (commit 1, 2).
  • Fix logic to keep ZBigFileH in sync with ZODB connection (commit).
  • Fix crash on PyVMA deallocation (commit).
  • Move py.bench to pygolang so that it can be used not only in Wendelin.core (commit).
  • Enhance t/qemu-runlinux - utility that is used to work on combined kernel/user-space workloads (commit 1, 2, 3, 4, 5, 6). This was in particular useful to develop Linux kernel fixes that are needed for Wendelin.core 2.0 (kernel commit 1, 2, 3, 4, 5, 6, 7).
  • Various bugfixes.

0.12 (2018-04-16)

  • Update licensing to be in line with whole Nexedi stack (commit). Please see https://www.nexedi.com/licensing for details, rationale and options.
  • Add ArrayRef utility to find out for a NumPy array its top-level root parent and how to recreate the array as some view of the root; this builds the foundation for e.g. sending arrays as references without copy in CMFActivity joblib backend (commit 1, 2, 3).
  • Don't crash if during loadblk() garbage collection was run twice at tricky times (commit 1, 2).
  • Don't crash on writeout if previously storeblk() resulted in error (commit).
  • Fix py.bench and rework it to produce output in Go benchmarking format (commit 1, 2, 3, 4, 5); add benchmarks for handling pagefaults (commit).
  • Use zodbtools/zodburi, if available, to open database by URL (commit).
  • Start to make sure it works with ZODB5 too (commit 1, 2).
  • Various bugfixes.

0.11 (2017-03-28)

  • Switch back to using ZBlk0 format by default (commit)

0.10 (2017-03-16)

  • Tell the world dtype=object is not supported (commit)

0.9 (2017-01-17)

  • Avoid deadlocks via doing storeblk() calls with virtmem lock released (commit 1, 2)
  • Don't crash if in loadblk() implementation an exception is internally raised & caught (commit 1, 2, 3)

0.8 (2016-09-28)

  • Do not leak memory when loading data in ZBlk1 format (commit).

0.7 (2016-07-14)

  • Add support for Python 3.5 (commit 1, 2)
  • Fix bug in pagemap code which could lead to crashes and other issues (commit)
  • Various bugfixes

0.6 (2016-06-13)

  • Add support for FORTRAN ordering (commit 1, 2)
  • Avoid deadlocks via doing loadblk() calls with virtmem lock released (commit 1, 2)
  • Various bugfixes

0.5 (2015-10-02)

  • Introduce another storage format, which is optimized for small changes, and make it the default. (commit 1, 2)
  • Various bugfixes and documentation improvements

0.4 (2015-08-19)

  • Add support for O(δ) in-place BigArray.append() (commit)
  • Implement proper multithreading support (commit)
  • Implement proper RAM pages invalidation when backing ZODB objects are changed from outside (commit 1, 2)
  • Fix all kind of failures that could happen when ZODB connection changes worker thread in-between handling requests (commit 1, 2)
  • Tox tests now cover usage with FileStorage, ZEO and NEO ZODB storages (commit 1, 2)
  • Various bugfixes

0.3 (2015-06-12)

  • Add support for automatic BigArray -> ndarray conversion, so that e.g. the following:

    A = BigArray(...)
numpy.mean(A)       # passing BigArray to plain NumPy function

    either succeeds, or raises MemoryError if not enough address space is available to cover whole A. (current limitation is ~ 127TB on linux/amd64)

    (commit)

  • Various bugfixes (build-fixes, crashes, overflows, etc)

0.2 (2015-05-25)

  • Add support for O(1) in-place BigArray.resize() (commit)
  • Various build bugfixes (older systems, non-std python, etc)

0.1 (2015-04-03)

  • Initial release