Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions tools/lucee-extensions/sqlite/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
dist/
temp/
108 changes: 108 additions & 0 deletions tools/lucee-extensions/sqlite/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# Wheels SQLite Lucee Extension

A Lucee CFML extension (`.lex`) that ships the SQLite JDBC driver as an OSGi bundle, so Lucee 7 can resolve `org.sqlite.JDBC` datasources without manual JAR drops.

## Why this exists

`wheels new` writes a SQLite-by-default datasource into `config/app.cfm`. Lucee 7 loads JDBC drivers via OSGi bundles in `lucee-server/bundles/`, but the stock Lucee Express bundles directory ships drivers for MySQL, MSSQL, PostgreSQL, and HSQLDB — **not** SQLite. Result: every fresh `wheels new` app fails on the first `wheels migrate latest` with:

```
The OSGi Bundle with name [org.xerial.sqlite-jdbc] is not available locally
[ (~/.wheels/servers/<name>/lucee-server/bundles)] or from the update provider
[ (https://update.lucee.org)].
```

This is finding **F8** in the fresh-VM onboarding journals. Until now the workaround was to hand-copy the JAR from `~/.wheels/express/<version>/lib/ext/sqlite-jdbc-*.jar` into `lucee-server/bundles/` — which works once but evaporates on every `wheels server start --force`.

## What's in the box

```
src/
build.properties Extension metadata (uuid, label, connection string, ...)
SQLite.cfc Lucee admin UI driver descriptor
sqlite-jdbc-3.49.1.0.jar Vendored xerial sqlite-jdbc JAR (Maven Central)
build.sh Build the .lex artifact
install.sh Install into a Lucee server's bundles/ directory
dist/ Build output: <bundle-symbolic-name>-<version>.lex
```

## Build

```bash
bash tools/lucee-extensions/sqlite/build.sh
```

Produces `dist/org.xerial.sqlite-jdbc-3.49.1.0.lex` (~14 MB). The build script:

1. Reads `src/build.properties` for extension UUID + JDBC metadata.
2. Reads OSGi headers from the bundled JAR (`Bundle-SymbolicName`, `Bundle-Version`).
3. Reads the JDBC driver class from `META-INF/services/java.sql.Driver` inside the JAR.
4. **Patches the JAR's MANIFEST.MF** (see "OSGi compatibility patch" below).
5. Stages `META-INF/MANIFEST.MF`, `jars/<jar>.jar`, `context/admin/dbdriver/SQLite.cfc`.
6. Zips → `.lex`.

## OSGi compatibility patch

The upstream `org.xerial:sqlite-jdbc` JAR is a valid OSGi bundle, but its manifest declares two things that block loading on Lucee 7 + JDK 11+:

```
Bundle-SymbolicName: org.xerial.sqlite-jdbc;singleton:=true
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
```

The strict `version=1.8` (exact match in some OSGi resolvers) prevents Felix on Java 21 from satisfying the requirement, even though Java 21 ⊇ Java 1.8. PostgreSQL's bundle uses `(version>=1.8)` and works; the sqlite-jdbc maintainers haven't relaxed this.

`build.sh` rewrites both headers in-place:

```
Bundle-SymbolicName: org.xerial.sqlite-jdbc
Require-Capability: osgi.ee;filter:="(&(|(osgi.ee=J2SE)(osgi.ee=JavaSE))(version>=1.8))"
```

(The patch is applied to a copy — the original `src/sqlite-jdbc-*.jar` is untouched.)

## Install

### Option A (recommended today): direct bundle install

```bash
bash tools/lucee-extensions/sqlite/install.sh ~/.wheels/servers/<name>
```

Drops the patched bundle straight into `<server>/lucee-server/bundles/`. Restart the server and SQLite datasources resolve.

### Option B: drop the .lex into Lucee's deploy folder

```bash
cp dist/org.xerial.sqlite-jdbc-3.49.1.0.lex ~/.wheels/servers/<name>/lucee-server/deploy/
```

This is the canonical Lucee install path (Lucee polls `deploy/` every 60s). On Lucee 7.0.0.395 we observed silent rejection — the file moves to `deploy/failed-to-deploy/` without an error log. The bundle path (Option A) is deterministic; the deploy path is the future-proof distribution mechanism once we (or upstream) figure out why Lucee 7 quarantines the file.

### Option C: install via Lucee admin UI

Log into `http://localhost:<port>/lucee/admin.cfm`, go to **Server → Extensions → Applications**, upload `dist/org.xerial.sqlite-jdbc-3.49.1.0.lex`. Same end state as Option A.

## Verify

```bash
# Server must already be running with a SQLite datasource configured.
curl -s http://localhost:9988/dbtest.cfm

# Before install: FAIL ... org.xerial.sqlite-jdbc is not available locally
# After install: OK datasource=sqliteapp result=1
```

## Upstream upgrade plan

1. Ship this artifact alongside Wheels releases (GitHub release asset on `wheels-dev/wheels`).
2. Have `wheels new` (or first `wheels start`) auto-run `install.sh` against the freshly-created server so SQLite-by-default works zero-config.
3. After 1–2 release cycles, file an upstream PR as `lucee/extension-jdbc-sqlite` so Lucee's update server distributes it. Once accepted, Wheels can drop the local bundle approach.
4. Separately, file an issue with `xerial/sqlite-jdbc` to relax their `Require-Capability: osgi.ee;version=1.8` to `version>=1.8` — eliminates the manifest-patch step and benefits everyone running OSGi.

## Trivia

- **Extension UUID**: `EACF838C-62B6-469C-A8DA-F802A9596C57` (regenerate if forking).
- **Bundle Symbolic Name**: `org.xerial.sqlite-jdbc` (matches `bundleName` in [cli/src/models/EnvironmentService.cfc:1009](../../../cli/src/models/EnvironmentService.cfc:1009) — already correct in Wheels CLI).
- **JDBC class**: `org.sqlite.JDBC` (declared by xerial's `META-INF/services/java.sql.Driver`).
- **Layout mirrors**: [lucee/extension-jdbc-postgresql](https://github.com/lucee/extension-jdbc-postgresql), [lucee/extension-jdbc-duckdb](https://github.com/lucee/extension-jdbc-duckdb).
223 changes: 223 additions & 0 deletions tools/lucee-extensions/sqlite/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
#!/usr/bin/env bash
# tools/lucee-extensions/sqlite/build.sh — build the SQLite Lucee extension (.lex)
#
# Mirrors the layout produced by lucee/extension-jdbc-postgresql and
# lucee/extension-jdbc-duckdb but uses bash+zip instead of Ant.
#
# Output: dist/<bundle-symbolic-name>-<bundle-version>.lex
# That .lex can be dropped into any Lucee server's lucee-server/deploy/
# directory; Lucee picks it up on next boot, installs the OSGi bundle
# into lucee-server/bundles/, and registers SQLite in the admin UI.

set -euo pipefail

HERE="$(cd "$(dirname "$0")" && pwd)"
SRC="$HERE/src"
DIST="$HERE/dist"
TEMP="$HERE/temp"

# ── Inputs ────────────────────────────────────────────────────────────

JAR="$(ls "$SRC"/*.jar 2>/dev/null | head -1)"
if [ -z "$JAR" ] || [ ! -f "$JAR" ]; then
echo "✗ no .jar found in $SRC/" >&2
echo " drop the xerial sqlite-jdbc JAR into src/ and re-run." >&2
exit 1
fi
JAR_NAME="$(basename "$JAR")"

CFC="$(ls "$SRC"/*.cfc 2>/dev/null | head -1)"
if [ -z "$CFC" ] || [ ! -f "$CFC" ]; then
echo "✗ no .cfc found in $SRC/" >&2
exit 1
fi
CFC_NAME="$(basename "$CFC")"

if [ ! -f "$SRC/build.properties" ]; then
echo "✗ src/build.properties is missing" >&2
exit 1
fi

# ── Read extension metadata from build.properties ─────────────────────

# shellcheck disable=SC2002
prop() {
local key="$1"
cat "$SRC/build.properties" \
| grep -E "^${key}=" \
| head -1 \
| sed -E "s/^${key}=//"
}
EXT_ID="$(prop id)"
EXT_LABEL="$(prop label)"
JDBC_ID="$(prop jdbcid)"
CONN_STR="$(prop connstr)"
EXT_DESC="$(prop description)"
LUCEE_CORE_VERSION="$(prop lucee-core-version)"

for v in EXT_ID EXT_LABEL JDBC_ID CONN_STR EXT_DESC LUCEE_CORE_VERSION; do
if [ -z "${!v}" ]; then
echo "✗ src/build.properties is missing key for $v" >&2
exit 1
fi
done

# ── Read OSGi bundle metadata from JAR's MANIFEST.MF ──────────────────

manifest_value() {
# OSGi manifests wrap long lines with a leading space on the next line.
# Unwrap first, then grep the header.
unzip -p "$JAR" META-INF/MANIFEST.MF \
| awk 'BEGIN{prev=""} /^ / {prev=prev substr($0,2); next} {if(prev!="")print prev; prev=$0} END{if(prev!="")print prev}' \
| grep -E "^$1:" \
| head -1 \
| sed -E "s/^$1:[ ]*//" \
| tr -d '\r'
}

SYMBOLIC_NAME_RAW="$(manifest_value Bundle-SymbolicName)"
# Strip OSGi directives like ";singleton:=true"
SYMBOLIC_NAME="${SYMBOLIC_NAME_RAW%%;*}"
BUNDLE_VERSION="$(manifest_value Bundle-Version)"

if [ -z "$SYMBOLIC_NAME" ] || [ -z "$BUNDLE_VERSION" ]; then
echo "✗ $JAR_NAME does not declare Bundle-SymbolicName / Bundle-Version" >&2
echo " This script needs an OSGi-ready JAR. Wrap with bnd if necessary." >&2
exit 1
fi

# ── Read JDBC driver class from the SPI file ──────────────────────────

DRIVER_CLASS="$(unzip -p "$JAR" META-INF/services/java.sql.Driver 2>/dev/null \
| grep -v '^[[:space:]]*#' \
| grep -v '^[[:space:]]*$' \
| head -1 \
| tr -d '[:space:]')"

if [ -z "$DRIVER_CLASS" ]; then
echo "✗ $JAR_NAME does not declare a java.sql.Driver SPI" >&2
exit 1
fi

echo "Extension metadata:"
echo " id = $EXT_ID"
echo " label = $EXT_LABEL"
echo " jdbcId = $JDBC_ID"
echo " connStr = $CONN_STR"
echo " driver class = $DRIVER_CLASS"
echo " bundle name = $SYMBOLIC_NAME"
echo " bundle version = $BUNDLE_VERSION"
echo " lucee-core min = $LUCEE_CORE_VERSION"

# ── Stage the .lex tree ───────────────────────────────────────────────

rm -rf "$TEMP" "$DIST"
STAGE="$TEMP/extension/$SYMBOLIC_NAME-$BUNDLE_VERSION"
mkdir -p "$STAGE/META-INF" "$STAGE/jars" "$STAGE/context/admin/dbdriver"

NOW="$(date '+%Y-%m-%d %H:%M:%S')"
JDBC_JSON="[{'label':'$EXT_LABEL','id':'$JDBC_ID','connectionString':'$CONN_STR','class':'$DRIVER_CLASS','bundleName':'$SYMBOLIC_NAME','bundleVersion':'$BUNDLE_VERSION'}]"

cat > "$STAGE/META-INF/MANIFEST.MF" <<EOF
Manifest-Version: 1.0
Built-Date: $NOW
version: "$BUNDLE_VERSION"
id: "$EXT_ID"
name: "$EXT_LABEL"
description: "$EXT_DESC"
category: "Datasource"
lucee-core-version: "$LUCEE_CORE_VERSION"
start-bundles: false
jdbc: "$JDBC_JSON"
EOF

# Copy + patch the JAR. Two manifest fixes are required for Lucee 7 / Felix on
# JDK 11+:
# 1. xerial declares Require-Capability with `version=1.8` (exact match) —
# Felix on Java 21 reports osgi.ee=JavaSE versions 1.8/11/21 separately,
# and exact-match against 1.8 fails resolution on some configurations.
# Relax to `(|(osgi.ee=J2SE)(osgi.ee=JavaSE))(version>=1.8)` so the
# bundle loads on any Java 8+ runtime — same filter PostgreSQL uses.
# 2. xerial uses `Bundle-SymbolicName: ...;singleton:=true`. The directive
# is harmless but the bundle filename / lookup we declare in the .lex
# manifest uses the bare symbolic name — strip the directive from the
# JAR's manifest so they match exactly.
# Renaming the file to <symbolicName>-<bundleVersion>.jar matches how Lucee's
# bundle loader expects to find it.
PATCHED_DIR="$TEMP/patched-jar"
mkdir -p "$PATCHED_DIR"
unzip -q "$JAR" -d "$PATCHED_DIR"
python3 - "$PATCHED_DIR/META-INF/MANIFEST.MF" "$SYMBOLIC_NAME" <<'PYEOF'
import sys
mf_path, symbolic_name = sys.argv[1], sys.argv[2]
with open(mf_path, 'rb') as f:
text = f.read().decode('utf-8')
def unfold(s):
lines, cur = [], ''
for line in s.split('\r\n'):
if line.startswith(' '):
cur += line[1:]
else:
if cur != '':
lines.append(cur)
cur = line
if cur != '':
lines.append(cur)
return lines
def fold(lines):
out = []
for line in lines:
if len(line) <= 70:
out.append(line)
continue
out.append(line[:70])
rest = line[70:]
while rest:
out.append(' ' + rest[:69])
rest = rest[69:]
return out
patched = []
for line in unfold(text):
if line.startswith('Bundle-SymbolicName:'):
patched.append(f'Bundle-SymbolicName: {symbolic_name}')
elif line.startswith('Require-Capability:'):
patched.append('Require-Capability: osgi.ee;filter:="(&(|(osgi.ee=J2SE)(osgi.ee=JavaSE))(version>=1.8))"')
else:
patched.append(line)
out = '\r\n'.join(fold(patched)) + '\r\n\r\n'
with open(mf_path, 'w', encoding='utf-8', newline='') as f:
f.write(out)
PYEOF

PATCHED_JAR="$STAGE/jars/$SYMBOLIC_NAME-$BUNDLE_VERSION.jar"
(
cd "$PATCHED_DIR"
zip -qrX0 "$PATCHED_JAR" META-INF/MANIFEST.MF
zip -qrX "$PATCHED_JAR" . -x "META-INF/MANIFEST.MF"
)

# Copy the admin driver descriptor.
cp "$CFC" "$STAGE/context/admin/dbdriver/$CFC_NAME"

# Copy logo if present.
if [ -f "$SRC/logo.png" ]; then
cp "$SRC/logo.png" "$STAGE/META-INF/logo.png"
fi

# ── Zip → .lex ────────────────────────────────────────────────────────

mkdir -p "$DIST"
LEX_NAME="$SYMBOLIC_NAME-$BUNDLE_VERSION.lex"
LEX_PATH="$DIST/$LEX_NAME"

(cd "$STAGE" && zip -qr "$LEX_PATH" .)

rm -rf "$TEMP"

echo ""
echo "✓ built $LEX_NAME"
echo " $LEX_PATH ($(du -h "$LEX_PATH" | awk '{print $1}'))"
echo ""
echo "Install:"
echo " cp \"$LEX_PATH\" ~/.wheels/servers/<name>/lucee-server/deploy/"
echo " wheels stop && wheels server start --force"
Loading