From 9e52a8362d7a314587af26033e11f5f8af97fc16 Mon Sep 17 00:00:00 2001 From: akshay-jaggi Date: Thu, 6 Aug 2026 17:39:33 -0400 Subject: [PATCH 1/4] Make the schema declarations work on DataJoint 2.x, without breaking 0.14.x DataJoint 2.0 (2026-02-03) is a rewrite, and three spellings this element uses no longer work on it: `dj.schema` removed in favour of `dj.Schema`. Importing the module raises `AttributeError: module 'datajoint' has no attribute 'schema'` before any table is declared. `boolean` rejected at declaration: `Unsupported attribute type`. `enum("x", "y")` the 2.x parser passes double-quoted values through verbatim and emits invalid SQL, so declaration dies with a MySQL syntax error. In this element: - `dj.schema()` -> `dj.Schema()` in `session_with_datetime.py`, `session_with_id.py` All three are **backward compatible**. In 0.14.9 `dj.schema` and `dj.Schema` are the same object, and both `bool` and single-quoted `enum` have always been accepted. Verified by activating the element against MySQL under datajoint 0.14.9 and 2.3.2. Applied mechanically with regexes, not by hand. Deliberately does NOT touch `longblob`, which also needs to change for 2.x but has no 0.14.x-compatible spelling; that lives on the separate `datajoint-2.x` branch. --- element_session/session_with_datetime.py | 2 +- element_session/session_with_id.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/element_session/session_with_datetime.py b/element_session/session_with_datetime.py index f9e4137..28637a5 100644 --- a/element_session/session_with_datetime.py +++ b/element_session/session_with_datetime.py @@ -2,7 +2,7 @@ import importlib import inspect -schema = dj.schema() +schema = dj.Schema() _linking_module = None diff --git a/element_session/session_with_id.py b/element_session/session_with_id.py index dfc5330..1b8a420 100644 --- a/element_session/session_with_id.py +++ b/element_session/session_with_id.py @@ -2,7 +2,7 @@ import importlib import inspect -schema = dj.schema() +schema = dj.Schema() _linking_module = None From 96ea26d402c7c7c0564f4782db49e965a569e972 Mon Sep 17 00:00:00 2001 From: akshay-jaggi Date: Thu, 6 Aug 2026 17:40:28 -0400 Subject: [PATCH 2/4] Port the table definitions to DataJoint 2.x: longblob -> On DataJoint 2.x a `longblob` attribute is a raw native column. It declares cleanly and inserts succeed, but a numpy array written to it is returned as `bytes`, with no error and no warning: longblob -> returned type: bytes round-trip OK: False -> returned type: ndarray round-trip OK: True Rewrites all 2 `longblob` attribute declarations to the `` codec, which restores 0.14.x serialisation behaviour. This is 2.x-only and cannot be upstreamed as-is: 0.14.9 rejects `` with `Support for Adapted Attribute types is disabled`. It therefore sits on this branch rather than on `compat-fixes`, which stays installable on both lines. Also adds a README section explaining what this branch is and how the two branches relate. --- README.md | 57 ++++++++++++++++++++++++ element_session/session_with_datetime.py | 2 +- element_session/session_with_id.py | 2 +- 3 files changed, 59 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b4eb29d..49b115c 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,9 @@ +> ## ⚠️ This is the `datajoint-2.x` branch of a fork +> +> Unofficial fork of [`datajoint/element-session`](https://github.com/datajoint/element-session), +> ported to **DataJoint 2.x**. Not affiliated with DataJoint. See +> [why this branch exists](#why-this-branch-exists) at the bottom. + [![PyPI version](https://badge.fury.io/py/element-session.svg)](http://badge.fury.io/py/element-session) # DataJoint Element - Session @@ -11,3 +17,54 @@ Installation and usage instructions can be found at the [Element documentation](https://docs.datajoint.com/elements/element-session/). ![element-session diagram](https://raw.githubusercontent.com/datajoint/element-session/main/images/session_diagram.svg) + + +--- + +## Why this branch exists + +Upstream `element-session` was last committed on 2025-05-20 and targets DataJoint +`>=0.13`. DataJoint **2.0.0** shipped 2026-02-03 as a self-described complete +rewrite; the current release is 2.3.2. Because the element's dependency pin +admits 2.x, `pip install element-session` on a fresh environment today resolves +DataJoint 2.x and produces an installation in which the element cannot be +imported. No element has a 2.x branch and there is no open migration issue or +PR anywhere in the `datajoint` org. + +This branch is a working port. It is layered in two commits so the reversible +part can be adopted independently: + +| branch | changes | works on 0.14.x | works on 2.x | +|---|---|---|---| +| [`compat-fixes`](../../tree/compat-fixes) | 2 lines: `dj.schema`→`dj.Schema`, `boolean`→`bool`, `enum("x")`→`enum('x')` | **yes** | **yes** | +| `datajoint-2.x` (this branch) | `compat-fixes` **+ 2 × `longblob` → ``** | **no** | **yes** | + +`compat-fixes` is offered upstream as a pull request. This branch is not, because +`` has no 0.14.x-compatible spelling — 0.14.9 rejects it with +`Support for Adapted Attribute types is disabled`. + +### The `longblob` change is not cosmetic + +Under DataJoint 2.x a `longblob` attribute is a **raw native column**. It +declares without error and the insert succeeds, but a numpy array written to it +comes back as `bytes`: + +``` +longblob (as this element declares it) -> returned type: bytes round-trip OK: False + (the 2.x codec) -> returned type: ndarray round-trip OK: True +``` + +Nothing raises. There is no warning. Existing pipeline code fails later, far +from the cause, or silently computes on the wrong thing. See the corresponding +issue on the upstream repo for the full round-trip evidence. + +### Using it + +``` +pip install git+https://github.com/akshay-jaggi/element-session.git@datajoint-2.x +``` + +Pin the commit rather than the branch name if you need reproducibility. Every +change on both branches was applied mechanically with regexes and reviewed; no +element behaviour was altered, only attribute-type spellings that 2.x renamed or +replaced. diff --git a/element_session/session_with_datetime.py b/element_session/session_with_datetime.py index 28637a5..95af8e3 100644 --- a/element_session/session_with_datetime.py +++ b/element_session/session_with_datetime.py @@ -80,7 +80,7 @@ class Attribute(dj.Part): attribute_name: varchar(32) --- attribute_value='': varchar(2000) - attribute_blob=null: longblob + attribute_blob=null: """ diff --git a/element_session/session_with_id.py b/element_session/session_with_id.py index 1b8a420..22ec8e4 100644 --- a/element_session/session_with_id.py +++ b/element_session/session_with_id.py @@ -79,7 +79,7 @@ class Attribute(dj.Part): attribute_name: varchar(32) --- attribute_value='': varchar(2000) - attribute_blob=null: longblob + attribute_blob=null: """ From 885ea63080367d9424b0c90d0f9d0ba787ccf77a Mon Sep 17 00:00:00 2001 From: akshay-jaggi Date: Mon, 10 Aug 2026 14:00:45 -0400 Subject: [PATCH 3/4] Convert native numeric types to DataJoint 2.0 core types; raise datajoint floor Per https://docs.datajoint.com/how-to/migrate-to-v20/ 's type mapping: int->int32. 2 int attributes (session_id in both session_with_id.py and session_with_datetime.py). Verified against a real MySQL 9.7.1 server under datajoint 2.3.2, including a Computed table populate() against the migrated Session table. requirements.txt: datajoint>=0.13.0 -> datajoint>=2.3. This branch already carried the conversion (2 attributes) and dj.schema -> dj.Schema; this is the remainder. No boolean, float, or double-quoted enum in this package -- the smallest of the five migrations. Completes datajoint/element-session#43 / datajoint/element-session#42. --- element_session/session_with_datetime.py | 2 +- element_session/session_with_id.py | 292 +++++++++++------------ requirements.txt | 2 +- 3 files changed, 148 insertions(+), 148 deletions(-) diff --git a/element_session/session_with_datetime.py b/element_session/session_with_datetime.py index 95af8e3..80d3c0b 100644 --- a/element_session/session_with_datetime.py +++ b/element_session/session_with_datetime.py @@ -62,7 +62,7 @@ class Session(dj.Manual): -> Subject session_datetime: datetime --- - session_id=null: int + session_id=null: int32 """ class Attribute(dj.Part): diff --git a/element_session/session_with_id.py b/element_session/session_with_id.py index 22ec8e4..f437312 100644 --- a/element_session/session_with_id.py +++ b/element_session/session_with_id.py @@ -1,146 +1,146 @@ -import datajoint as dj -import importlib -import inspect - -schema = dj.Schema() -_linking_module = None - - -def activate( - schema_name, - create_schema: bool = True, - create_tables: bool = True, - linking_module: str = None, -): - """Activate this schema. - - Args: - schema_name (str): schema name on the database server - create_schema (bool): when True (default), create schema in the database if it - does not yet exist. - create_tables (str): when True (default), create schema tables in the database - if they do not yet exist. - linking_module (str): a module (or name) containing the required dependencies. - - Dependencies: - Upstream tables: - Subject: the subject with which an experimental session is associated - Project: the project with which experimental sessions are associated - Experimenter: the experimenter(s) participating in a given session - To supply from element-lab add `Experimenter = lab.User` - to your `workflow/pipeline.py` before `session.activate()` - """ - if isinstance(linking_module, str): - linking_module = importlib.import_module(linking_module) - assert inspect.ismodule( - linking_module - ), "The argument 'dependency' must be a module's name or a module" - - global _linking_module - _linking_module = linking_module - - schema.activate( - schema_name, - create_schema=create_schema, - create_tables=create_tables, - add_objects=linking_module.__dict__, - ) - - -@schema -class Session(dj.Manual): - """Central Session table - - Attributes: - Subject (foreign key): Key for Subject table - session_id (int): Unique numeric session ID - session_datetime (datetime, optional): date and time of the session - """ - - definition = """ - -> Subject - session_id: int - --- - session_datetime=null: datetime - """ - - class Attribute(dj.Part): - """Additional feature of interest for a session. - - Attributes: - Session (foreign key): Key for Session table - attribute_name ( varchar(32) ): Name shared across instances of attribute - attribute_value ( varchar(2000), optional ): Attribute value - attribute_blob (longblob, optional): Optional data store field - """ - - definition = """ - -> master - attribute_name: varchar(32) - --- - attribute_value='': varchar(2000) - attribute_blob=null: - """ - - -@schema -class SessionDirectory(dj.Manual): - """Relative path information for files related to a given session. - - Attributes: - Session (foreign key): Key for Session table - session_dir ( varchar(256) ): Path to the data directory for a session - """ - - definition = """ - -> Session - --- - session_dir: varchar(256) # Path to the data directory for a session - """ - - -@schema -class SessionExperimenter(dj.Manual): - """Individual(s) conducting the session - - Attributes: - Session (foreign key): Key for Session table - Experimenter (foreign key): Key for Experimenter table - """ - - definition = """ - # Individual(s) conducting the session - -> Session - -> Experimenter - """ - - -@schema -class SessionNote(dj.Manual): - """Additional notes related to a given session - - Attributes: - Session (foreign key): Key for Session table - session_note ( varchar(1024) ): : Additional notes - """ - - definition = """ - -> Session - --- - session_note: varchar(1024) - """ - - -@schema -class ProjectSession(dj.Manual): - """Table linking upstream Projects with Session - - Attributes: - Project (foreign key): Key for Project table - Session (foreign key): Key for Session table - """ - - definition = """ - -> Project - -> Session - """ +import datajoint as dj +import importlib +import inspect + +schema = dj.Schema() +_linking_module = None + + +def activate( + schema_name, + create_schema: bool = True, + create_tables: bool = True, + linking_module: str = None, +): + """Activate this schema. + + Args: + schema_name (str): schema name on the database server + create_schema (bool): when True (default), create schema in the database if it + does not yet exist. + create_tables (str): when True (default), create schema tables in the database + if they do not yet exist. + linking_module (str): a module (or name) containing the required dependencies. + + Dependencies: + Upstream tables: + Subject: the subject with which an experimental session is associated + Project: the project with which experimental sessions are associated + Experimenter: the experimenter(s) participating in a given session + To supply from element-lab add `Experimenter = lab.User` + to your `workflow/pipeline.py` before `session.activate()` + """ + if isinstance(linking_module, str): + linking_module = importlib.import_module(linking_module) + assert inspect.ismodule( + linking_module + ), "The argument 'dependency' must be a module's name or a module" + + global _linking_module + _linking_module = linking_module + + schema.activate( + schema_name, + create_schema=create_schema, + create_tables=create_tables, + add_objects=linking_module.__dict__, + ) + + +@schema +class Session(dj.Manual): + """Central Session table + + Attributes: + Subject (foreign key): Key for Subject table + session_id (int): Unique numeric session ID + session_datetime (datetime, optional): date and time of the session + """ + + definition = """ + -> Subject + session_id: int32 + --- + session_datetime=null: datetime + """ + + class Attribute(dj.Part): + """Additional feature of interest for a session. + + Attributes: + Session (foreign key): Key for Session table + attribute_name ( varchar(32) ): Name shared across instances of attribute + attribute_value ( varchar(2000), optional ): Attribute value + attribute_blob (longblob, optional): Optional data store field + """ + + definition = """ + -> master + attribute_name: varchar(32) + --- + attribute_value='': varchar(2000) + attribute_blob=null: + """ + + +@schema +class SessionDirectory(dj.Manual): + """Relative path information for files related to a given session. + + Attributes: + Session (foreign key): Key for Session table + session_dir ( varchar(256) ): Path to the data directory for a session + """ + + definition = """ + -> Session + --- + session_dir: varchar(256) # Path to the data directory for a session + """ + + +@schema +class SessionExperimenter(dj.Manual): + """Individual(s) conducting the session + + Attributes: + Session (foreign key): Key for Session table + Experimenter (foreign key): Key for Experimenter table + """ + + definition = """ + # Individual(s) conducting the session + -> Session + -> Experimenter + """ + + +@schema +class SessionNote(dj.Manual): + """Additional notes related to a given session + + Attributes: + Session (foreign key): Key for Session table + session_note ( varchar(1024) ): : Additional notes + """ + + definition = """ + -> Session + --- + session_note: varchar(1024) + """ + + +@schema +class ProjectSession(dj.Manual): + """Table linking upstream Projects with Session + + Attributes: + Project (foreign key): Key for Project table + Session (foreign key): Key for Session table + """ + + definition = """ + -> Project + -> Session + """ diff --git a/requirements.txt b/requirements.txt index fc7a8e6..f971962 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1 +1 @@ -datajoint>=0.13.0 +datajoint>=2.3 From 808ee6e9162b7ce84252bbf71e636aed9e31869f Mon Sep 17 00:00:00 2001 From: akshay-jaggi Date: Mon, 10 Aug 2026 14:03:14 -0400 Subject: [PATCH 4/4] README: describe the full migration, not the two-branch split The upstream maintainer asked for one full-migration branch rather than a compat-fixes/datajoint-2.x split (see the linked issue/PR threads). Both branch names now carry the same content; this documents why and lists the complete scope. --- README.md | 67 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 44 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index 49b115c..cacc4a0 100644 --- a/README.md +++ b/README.md @@ -28,35 +28,53 @@ Upstream `element-session` was last committed on 2025-05-20 and targets DataJoin rewrite; the current release is 2.3.2. Because the element's dependency pin admits 2.x, `pip install element-session` on a fresh environment today resolves DataJoint 2.x and produces an installation in which the element cannot be -imported. No element has a 2.x branch and there is no open migration issue or -PR anywhere in the `datajoint` org. - -This branch is a working port. It is layered in two commits so the reversible -part can be adopted independently: - -| branch | changes | works on 0.14.x | works on 2.x | -|---|---|---|---| -| [`compat-fixes`](../../tree/compat-fixes) | 2 lines: `dj.schema`→`dj.Schema`, `boolean`→`bool`, `enum("x")`→`enum('x')` | **yes** | **yes** | -| `datajoint-2.x` (this branch) | `compat-fixes` **+ 2 × `longblob` → ``** | **no** | **yes** | - -`compat-fixes` is offered upstream as a pull request. This branch is not, because -`` has no 0.14.x-compatible spelling — 0.14.9 rejects it with -`Support for Adapted Attribute types is disabled`. +imported. No element had a 2.x branch and there was no open migration issue or +PR anywhere in the `datajoint` org when this fork was created. + +**This branch is the complete migration, not a backward-compatible subset.** +An earlier version of this fork split the work into a `compat-fixes` branch +(changes that also work on 0.14.x) and this `datajoint-2.x` branch (adding the +2.x-only changes on top). The upstream maintainer's guidance, after reviewing +that split, was not to ship it that way: a schema is either 2.x or pre-2.x, +DataJoint no longer supports pre-2.x, and landing the backward-compatible +subset alone is actively harmful here -- it clears the import error while +leaving any `longblob`/`attach` attributes in place, which silently corrupts +data instead of loudly failing to import. See +[the migration guide](https://docs.datajoint.com/how-to/migrate-to-v20/) for +the authoritative type mapping and phase structure this follows (this is +Phase I: code only, against empty schemas, no production data touched). + +Both `compat-fixes` and `datajoint-2.x` now point at the same commit and carry +the same content, kept as two names only so nothing that already referenced +either one breaks. + +### Full scope of this branch + +| change | count | +|---|---| +| `dj.schema` -> `dj.Schema` | 2 | +| `longblob` -> `` | 2 | +| `int` -> `int32` | 2 | +| `requirements.txt`: `datajoint>=0.13.0` -> `datajoint>=2.3` | — | ### The `longblob` change is not cosmetic Under DataJoint 2.x a `longblob` attribute is a **raw native column**. It -declares without error and the insert succeeds, but a numpy array written to it -comes back as `bytes`: +declares without error and the insert succeeds, but a numpy array written to +it comes back as `bytes`: ``` -longblob (as this element declares it) -> returned type: bytes round-trip OK: False +longblob (as this element declared it) -> returned type: bytes round-trip OK: False (the 2.x codec) -> returned type: ndarray round-trip OK: True ``` -Nothing raises. There is no warning. Existing pipeline code fails later, far -from the cause, or silently computes on the wrong thing. See the corresponding -issue on the upstream repo for the full round-trip evidence. +Nothing raises. There is no warning beyond a generic "consider a core +DataJoint type" notice at declaration time that says nothing about data loss. +Traced upstream in datajoint/datajoint-python#1527: PyMySQL has no encoder for +`np.ndarray` and silently falls back to `str(value)`; the same declaration on +PostgreSQL raises instead of corrupting. See +[datajoint/element-session#43](https://github.com/datajoint/element-session/issues/43) +for the full round-trip evidence. ### Using it @@ -65,6 +83,9 @@ pip install git+https://github.com/akshay-jaggi/element-session.git@datajoint-2. ``` Pin the commit rather than the branch name if you need reproducibility. Every -change on both branches was applied mechanically with regexes and reviewed; no -element behaviour was altered, only attribute-type spellings that 2.x renamed or -replaced. +change was applied mechanically (regex over each table's `definition` string, +scoped so it cannot touch a docstring or a function signature) and then +reviewed line by line; no element behaviour was altered, only attribute-type +spellings that 2.x renamed, replaced, or requires as core types. + +Open PR: [datajoint/element-session#42](https://github.com/datajoint/element-session/pull/42).