Repository navigation
fix(analytics): count empty selections as none - #1155
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughChangesThe PR normalizes empty analytics selections to Analytics selection handling
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install failed. For unrecoverable errors, disable the tool in CodeRabbit configuration. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 79244fb5-037f-486a-8a30-b435f3abd73a
📒 Files selected for processing (8)
apps/cli/test/analytics.test.tsapps/web/src/app/(home)/analytics/_components/dev-environment-charts.tsxapps/web/src/app/(home)/analytics/_components/stack-configuration-charts.tsxpackages/backend/convex/analytics.tspackages/backend/convex/analytics_helpers.tspackages/backend/test/analytics-helpers.test.tspackages/types/src/analytics.tspackages/types/test/analytics.test.ts
| export const replaceSelectionDistributions = internalMutation({ | ||
| args: { | ||
| expectedTotal: v.number(), | ||
| frontend: distributionValidator, | ||
| addons: distributionValidator, | ||
| examples: distributionValidator, | ||
| }, | ||
| returns: v.null(), | ||
| handler: async (ctx, args) => { | ||
| const stats = await ctx.db.query("analyticsStats").first(); | ||
| if (!stats) { | ||
| throw new Error("Cannot rebuild missing analytics stats"); | ||
| } | ||
| if (stats.totalProjects !== args.expectedTotal) { | ||
| throw new Error( | ||
| `Expected ${args.expectedTotal} active events, stats contain ${stats.totalProjects}`, | ||
| ); | ||
| } | ||
|
|
||
| await ctx.db.patch(stats._id, { | ||
| frontend: args.frontend, | ||
| addons: args.addons, | ||
| examples: args.examples, | ||
| }); |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🌐 Web query:
In Convex, do separate ctx.runQuerycalls from aninternalActionand a laterctx.runMutation share a consistent snapshot or transaction? What concurrency control is required before replacing aggregates rebuilt from paginated reads?
💡 Result:
No, separate calls to ctx.runQuery or ctx.runMutation from an internalAction (or any action) do not share a consistent snapshot or transaction [1][2]. Each call to ctx.runQuery or ctx.runMutation executes as an independent, serializable transaction [1][2][3]. Consequently, they are not guaranteed to see the same state of the database; for instance, a sequence of calls may read different versions of the same document [1][2]. To ensure consistency across multiple database operations in an action, you should refactor the logic into a single internal query or mutation that performs all necessary reads and writes in one atomic transaction [1][2]. When rebuilding aggregates from paginated reads—such as when synchronizing an aggregate component with a source-of-truth table—you must manage concurrency control to avoid Optimistic Concurrency Control (OCC) conflicts and performance issues [4][5]. Because aggregates often use internal B-tree structures where operations on adjacent keys or aggregate nodes can interfere with one another, high-frequency writes or large-scale re-indexing can lead to contention [4][6]. Recommended practices for rebuilding or updating aggregates include: 1. Use Idempotent Operations: Use methods such as insertIfDoesNotExist, replaceOrInsert, or deleteIfExists rather than direct inserts or replacements [5][7]. This allows the aggregate state to reach consistency even if the migration or update process is interrupted or re-run [5][7]. 2. Batching and Pagination: When processing large datasets, use pagination to handle data in smaller, discrete batches rather than loading entire structures into memory [6]. 3. Minimized Contention: Be aware that reading an entire aggregate structure creates a broad read dependency that causes the operation to conflict with any mutation modifying that structure [4][5]. Use bounds to restrict the scope of your reads and writes wherever possible [6]. 4. Migration Patterns: For initial backfills, use a background migration that processes data in paginated batches, and ensure that live traffic writes are handled idempotently to converge on the correct state [5][7].
Citations:
- 1: https://docs.convex.dev/understanding/best-practices/
- 2: https://docs.convex.dev/functions/actions
- 3: https://stack.convex.dev/how-convex-works
- 4: https://mintlify.wiki/get-convex/aggregate/performance/read-dependencies
- 5: https://github.com/get-convex/aggregate
- 6: https://mintlify.wiki/get-convex/aggregate/operations/troubleshooting
- 7: https://github.com/get-convex/aggregate/blob/main/README.md
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- analytics.ts outline ---'
ast-grep outline packages/backend/convex/analytics.ts --view compact || true
printf '%s\n' '--- target sections ---'
sed -n '300,450p' packages/backend/convex/analytics.ts
printf '%s\n' '--- related symbols and call sites ---'
rg -n -C 4 'replaceSelectionDistributions|rebuildSelectionDistributions|analyticsStats|totalProjects|quarantine' packages/backend/convexRepository: AmanVarshney01/create-better-t-stack
Length of output: 25525
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- ingestion and repair mutations ---'
sed -n '1,125p' packages/backend/convex/analytics.ts
sed -n '430,490p' packages/backend/convex/analytics.ts
printf '%s\n' '--- helper update semantics ---'
sed -n '1,245p' packages/backend/convex/analytics_helpers.ts
printf '%s\n' '--- schema fields ---'
sed -n '35,65p' packages/backend/convex/schema.ts
printf '%s\n' '--- scheduled or external callers ---'
rg -n -C 5 'rebuildSelectionDistributions|quarantineInvalidEvents|quarantineInvalidEventsBatch|recordAnalytics|internal\.analytics' packages/backend packages --glob '!packages/backend/convex/analytics.ts'Repository: AmanVarshney01/create-better-t-stack
Length of output: 15670
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
# Read-only model of the active-event set used by the rebuild.
initial = {"A": "frontend-a", "B": "frontend-b"}
scanned = dict(initial)
# Concurrent mutations: ingest C, then quarantine A.
current = dict(initial)
current["C"] = "frontend-c"
del current["A"]
expected_total = len(scanned)
stats_total_after_mutations = len(current)
print({
"scanned_active_events": scanned,
"current_active_events": current,
"expectedTotal": expected_total,
"stats.totalProjects": stats_total_after_mutations,
"count_check_passes": expected_total == stats_total_after_mutations,
"distributions_equal": sorted(scanned.values()) == sorted(current.values()),
})
PYRepository: AmanVarshney01/create-better-t-stack
Length of output: 409
Protect replacement from concurrent analytics changes.
Separate ctx.runQuery calls and the later ctx.runMutation do not share one snapshot. An insertion followed by quarantine can keep stats.totalProjects equal to expectedTotal, so the check passes while the distributions describe a different active-event set.
Serialize analytics ingestion, quarantine, and rebuild operations. Alternatively, add a monotonic analytics-stats revision, compare it in replaceSelectionDistributions, and retry when it changes.
| const AnalyticsFrontendListSchema = z | ||
| .array(FrontendSchema) | ||
| .max(FRONTEND_VALUES.length) | ||
| .refine(hasUniqueValues) | ||
| .refine(hasNoMixedNone); | ||
| .refine(hasNoMixedNone) | ||
| .transform(normalizeAnalyticsSelection); | ||
|
|
||
| const AnalyticsAddonListSchema = z | ||
| .array(AnalyticsAddonSchema) | ||
| .max(ANALYTICS_ADDON_VALUES.length) | ||
| .refine(hasUniqueValues) | ||
| .refine(hasNoMixedNone); | ||
| .refine(hasNoMixedNone) | ||
| .transform(normalizeAnalyticsSelection); | ||
|
|
||
| const AnalyticsExampleListSchema = z | ||
| .array(ExamplesSchema) | ||
| .max(EXAMPLES_VALUES.length) | ||
| .refine(hasUniqueValues) | ||
| .refine(hasNoMixedNone); | ||
| .refine(hasNoMixedNone) | ||
| .transform(normalizeAnalyticsSelection); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
For Zod 4.4.3, does z.array(...).transform(fn).optional().parse(undefined)invokefn? Does z.array(...).optional().transform(fn).parse(undefined)invokefn?
💡 Result:
In Zod 4.4.3, both scenarios will invoke the transform function when passed undefined. 1. z.array(...).transform(fn).optional().parse(undefined): The transform function fn is invoked. In Zod 4, $ZodTransform is designed to run even when the input is undefined. To preserve the expected behavior where an outer .optional() results in undefined rather than an error or a transformed value, $ZodTransform attaches a fallback flag to the result [1]. When the outer ZodOptional encounters this fallback flag on a result derived from an undefined input, it short-circuits the process and returns undefined [1]. 2. z.array(...).optional().transform(fn).parse(undefined): The transform function fn is invoked. Because .optional() is applied before the .transform(fn), the input to the transform is undefined. Zod 4 transforms do not short-circuit on undefined by default; they process the undefined input, meaning the function fn will be executed with undefined as its argument [1]. In summary, placing .optional() after .transform() allows Zod to handle the undefined output via its internal fallback mechanism [1], whereas placing .optional() before .transform() means the transform function itself receives and must handle the undefined input [1].
Citations:
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- package files ---'
git ls-files 'packages/types/*' 'packages/types/test/*' | sed -n '1,120p'
printf '%s\n' '--- analytics source ---'
cat -n packages/types/src/analytics.ts | sed -n '1,180p'
printf '%s\n' '--- analytics tests ---'
cat -n packages/types/test/analytics.test.ts | sed -n '1,240p'
printf '%s\n' '--- zod declarations ---'
rg -n '"zod"|zod@|Zod' package.json packages/types package-lock.json pnpm-lock.yaml yarn.lock 2>/dev/null | sed -n '1,160p'Repository: AmanVarshney01/create-better-t-stack
Length of output: 10599
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- locked Zod version ---'
rg -n -C 3 '(^|/)(zod|`@zod/`)' pnpm-lock.yaml package-lock.json yarn.lock bun.lock bun.lockb 2>/dev/null | sed -n '1,160p' || true
printf '%s\n' '--- available local Zod sources ---'
find . -path '*/node_modules/zod*' -o -path '*/node_modules/@zod*' | sed -n '1,80p'
printf '%s\n' '--- relevant source references ---'
rg -n 'ZodOptional|\$ZodOptional|ZodTransform|\$ZodTransform|fallback' . --glob '!**/node_modules/**' --glob '!**/.git/**' | sed -n '1,160p'Repository: AmanVarshney01/create-better-t-stack
Length of output: 5673
🌐 Web query:
Zod v4.4.3 source $ZodOptional $ZodTransform undefined parse optional transform
💡 Result:
In Zod v4.4.3, the interaction between $ZodOptional and $ZodTransform is governed by a runtime "fallback" mechanism designed to maintain consistent undefined handling [1][2]. Core Mechanisms: - Fallback Payload: $ZodTransform stamps every invocation result with a fallback: true flag in the internal ParsePayload [1][2][3]. - Short-Circuiting: $ZodOptional checks this flag; when it encounters a fallback result from an inner transform while the original input was undefined, it "clobbers" the transformed output and short-circuits the parse to return undefined [1][2]. - Runtime Optionality: Since v4.4.0+, $ZodTransform (and by extension, z.preprocess) declares optin === "optional" at runtime [1][2]. This allows schemas with user-written input handlers (like transforms or preprocessors) to accept undefined even if the static type suggests otherwise [2]. Key Behaviors: - Preserved Stability: This design ensures that chains like z.preprocess(fn, schema).optional().parse(undefined) correctly return undefined, maintaining compatibility with long-standing Zod behaviors [1][2]. - Transformation and Key Optionality: While undefined input is handled, applying .transform() to a schema in Zod v4 generally changes the resulting type from "key-optional" (e.g., { prop?: T }) to "value-optional" (e.g., { prop: T | undefined }) [4]. This is an intentional design choice; to preserve key optionality in the output, you must explicitly omit the key from the returned object in your transform function [5][4]. Summary of Source Roles: - $ZodTransform: Responsible for the fallback flag, signaling that the output is provisional for undefined inputs [2][3]. - $ZodOptional: Responsible for consuming the fallback flag to ensure that undefined inputs are treated correctly by wrapping schemas [1][2]. - $ZodPreprocess: Structurally pins its optionality (optin) to the inner schema (via lazy metadata), ensuring that optionality placed inside a preprocessor correctly affects the parent object's parsing logic [6].
Citations:
- 1: fix(v4): generalize optin/fallback to transform; restore preprocess on absent keys colinhacks/zod#5941
- 2: colinhacks/zod@c2be4f8
- 3: https://github.com/colinhacks/zod/blob/912f0f51/packages/zod/src/v4/core/schemas.ts
- 4: Behaviour change from v3 to v4 with key optional properties colinhacks/zod#5178
- 5: How to have .optional() survive during transform() colinhacks/zod#3291
- 6: colinhacks/zod@02c2baf
🏁 Script executed:
#!/bin/bash
set -eu
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
curl -fsSL "https://raw.githubusercontent.com/colinhacks/zod/v4.4.3/packages/zod/src/v4/core/schemas.ts" \
-o "$tmpdir/schemas.ts"
printf '%s\n' '--- exact version source: optional and transform ---'
rg -n -C 18 'function handleOptionalResult|class.*Optional|\\$ZodOptional|\\$ZodTransform|fallback' "$tmpdir/schemas.ts" | sed -n '1,320p'
printf '%s\n' '--- runtime availability ---'
command -v node || true
command -v bun || true
command -v npm || trueRepository: AmanVarshney01/create-better-t-stack
Length of output: 11223
🏁 Script executed:
#!/bin/bash
set -eu
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
curl -fsSL "https://raw.githubusercontent.com/colinhacks/zod/v4.4.3/packages/zod/src/v4/core/schemas.ts" \
-o "$tmpdir/schemas.ts"
printf '%s\n' '--- optional parser ---'
sed -n '3474,3528p' "$tmpdir/schemas.ts"
printf '%s\n' '--- object property parser ---'
rg -n -C 20 'def.keyType|isOptional|optin|keyResult' "$tmpdir/schemas.ts" | sed -n '1,260p'
printf '%s\n' '--- package entrypoints and runtime files ---'
curl -fsSL "https://registry.npmjs.org/zod/4.4.3" | node -e '
let s=""; process.stdin.on("data",d=>s+=d).on("end",()=> {
const p=JSON.parse(s);
console.log(JSON.stringify({version:p.version,main:p.main,module:p.module,types:p.types,files:p.files}, null, 2));
})'Repository: AmanVarshney01/create-better-t-stack
Length of output: 11741
🏁 Script executed:
#!/bin/bash
set -eu
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
curl -fsSL "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz" | tar -xz -C "$tmpdir"
node - <<'JS' "$tmpdir/package"
const path = process.argv[2];
const { z } = require(path);
const calls = [];
const normalize = (value) => {
calls.push(value);
return value && value.length > 0 ? value : ["none"];
};
const transformedThenOptional = z.array(z.string()).transform(normalize).optional();
const optionalThenTransformed = z.array(z.string()).optional().transform(normalize);
const objectOuterOptional = z.object({ field: transformedThenOptional });
const objectInnerOptional = z.object({ field: optionalThenTransformed });
for (const [name, schema] of [
["transform().optional()", transformedThenOptional],
["optional().transform()", optionalThenTransformed],
["object outer optional", objectOuterOptional],
["object inner optional", objectInnerOptional],
]) {
calls.length = 0;
const result = schema.safeParse(name.startsWith("object") ? {} : undefined);
console.log(name, JSON.stringify({
success: result.success,
data: result.success ? result.data : undefined,
calls,
}));
}
JSRepository: AmanVarshney01/create-better-t-stack
Length of output: 436
Normalize missing selections inside each list schema.
The outer .optional() bypasses normalizeAnalyticsSelection for missing values. Missing frontend, addons, and examples values therefore remain undefined instead of becoming ["none"].
Apply .optional() before .transform(normalizeAnalyticsSelection) in each list schema. Remove the outer .optional() calls and add missing-field assertions to packages/types/test/analytics.test.ts.
Summary
noneoption at the shared schema and aggregate boundariesnoneis visibly defined as “created without” that optionRoot cause and data audit
The CLI historically sent
[]when no multi-select option was chosen, while the aggregate logic counted only array members. That made “none” nearly disappear from the charts even though raw events were valid.I audited all 56,004 active production events against every aggregate. Only the three multi-select distributions were inconsistent:
none: 2,174none: 2,938none: 42,165All other option distributions and control totals matched the raw events.
Production repair
frontend,addons, andexamplesaggregate mapsThe replacement mutation is guarded by the scanned active-event total so it cannot install a stale partial result if ingestion changes during the paginated scan.
Verification
bun run checkbun run build:webSummary by CodeRabbit
Analytics
Bug Fixes
Tests