Skip to content

fix(analytics): count empty selections as none - #1155

Merged
AmanVarshney01 merged 1 commit into
mainfrom
aman/fix-analytics-option-counts
Aug 12, 2026
Merged

AmanVarshney01 merged 1 commit into
mainfrom
aman/fix-analytics-option-counts

Conversation

@AmanVarshney01

@AmanVarshney01 AmanVarshney01 commented Aug 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • normalize empty frontend, addon, and example selections to the canonical none option at the shared schema and aggregate boundaries
  • add a guarded, paginated internal Convex repair action for the three affected distributions
  • clarify chart descriptions so none is visibly defined as “created without” that option

Root cause and data audit

The CLI historically sent [] when no multi-select option was chosen, while the aggregate logic counted only array members. That made “none” nearly disappear from the charts even though raw events were valid.

I audited all 56,004 active production events against every aggregate. Only the three multi-select distributions were inconsistent:

  • frontend none: 2,174
  • addons none: 2,938
  • examples none: 42,165

All other option distributions and control totals matched the raw events.

Production repair

  • exported a fresh production backup before mutation
  • ran the repair in dry-run mode first
  • rebuilt only frontend, addons, and examples aggregate maps
  • did not modify or delete raw analytics events
  • reconciled every aggregate control total to 56,004 afterward

The replacement mutation is guarded by the scanned active-event total so it cannot install a stale partial result if ingestion changes during the paginated scan.

Verification

  • bun run check
  • bun run build:web
  • Convex codegen and TypeScript validation
  • analytics type, backend-helper, and CLI unit tests
  • full CLI suite: 574 passed, 13 skipped, 0 failed
  • live production chart/data verification after the aggregate rebuild

Summary by CodeRabbit

  • Analytics

    • Improved reporting for projects created without a selected frontend, addon, or example.
    • Analytics charts now clearly include projects with no corresponding selection.
    • Empty or missing selections are consistently grouped as “none.”
  • Bug Fixes

    • Corrected analytics aggregation and distribution rebuilding for unselected configuration options.
  • Tests

    • Added coverage for empty and missing selections across analytics tracking and reporting.

@vercel

vercel Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
create-better-t-stack-web Ready Ready Preview Aug 12, 2026 11:28am

Request Review

@AmanVarshney01
AmanVarshney01 marked this pull request as ready for review August 12, 2026 11:28
@AmanVarshney01
AmanVarshney01 merged commit 403a331 into main Aug 12, 2026
3 checks passed
@AmanVarshney01
AmanVarshney01 deleted the aman/fix-analytics-option-counts branch August 12, 2026 11:30
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

The PR normalizes empty analytics selections to ["none"], applies this behavior during ingestion, adds internal distribution rebuild functions, updates chart descriptions, and adds regression tests.

Analytics selection handling

Layer / File(s) Summary
Selection normalization contract
packages/types/src/analytics.ts, packages/types/test/analytics.test.ts, apps/cli/test/analytics.test.ts
Selection schemas normalize empty or missing frontend, addon, and example values to ["none"]. Tests verify the normalized event shape.
Normalized analytics ingestion
packages/backend/convex/analytics_helpers.ts, packages/backend/test/analytics-helpers.test.ts
Analytics ingestion normalizes selections before primary-value selection and distribution updates. Tests verify none counts and stack combinations.
Distribution rebuild workflow
packages/backend/convex/analytics.ts
Internal functions paginate active events, aggregate distributions, validate totals, support dry-run execution, and replace stored distributions.
Analytics chart descriptions
apps/web/src/app/(home)/analytics/_components/dev-environment-charts.tsx, apps/web/src/app/(home)/analytics/_components/stack-configuration-charts.tsx
Chart descriptions now include projects created without a frontend, addon, or example.

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main analytics change: counting empty selections as the canonical none option.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint install failed. For unrecoverable errors, disable the tool in CodeRabbit configuration.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 79244fb5-037f-486a-8a30-b435f3abd73a

📥 Commits

Reviewing files that changed from the base of the PR and between 5a14231 and 99093d7.

📒 Files selected for processing (8)
  • apps/cli/test/analytics.test.ts
  • apps/web/src/app/(home)/analytics/_components/dev-environment-charts.tsx
  • apps/web/src/app/(home)/analytics/_components/stack-configuration-charts.tsx
  • packages/backend/convex/analytics.ts
  • packages/backend/convex/analytics_helpers.ts
  • packages/backend/test/analytics-helpers.test.ts
  • packages/types/src/analytics.ts
  • packages/types/test/analytics.test.ts

Comment on lines +342 to +365
export const replaceSelectionDistributions = internalMutation({
args: {
expectedTotal: v.number(),
frontend: distributionValidator,
addons: distributionValidator,
examples: distributionValidator,
},
returns: v.null(),
handler: async (ctx, args) => {
const stats = await ctx.db.query("analyticsStats").first();
if (!stats) {
throw new Error("Cannot rebuild missing analytics stats");
}
if (stats.totalProjects !== args.expectedTotal) {
throw new Error(
`Expected ${args.expectedTotal} active events, stats contain ${stats.totalProjects}`,
);
}

await ctx.db.patch(stats._id, {
frontend: args.frontend,
addons: args.addons,
examples: args.examples,
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🌐 Web query:

In Convex, do separate ctx.runQuerycalls from aninternalActionand a laterctx.runMutation share a consistent snapshot or transaction? What concurrency control is required before replacing aggregates rebuilt from paginated reads?

💡 Result:

No, separate calls to ctx.runQuery or ctx.runMutation from an internalAction (or any action) do not share a consistent snapshot or transaction [1][2]. Each call to ctx.runQuery or ctx.runMutation executes as an independent, serializable transaction [1][2][3]. Consequently, they are not guaranteed to see the same state of the database; for instance, a sequence of calls may read different versions of the same document [1][2]. To ensure consistency across multiple database operations in an action, you should refactor the logic into a single internal query or mutation that performs all necessary reads and writes in one atomic transaction [1][2]. When rebuilding aggregates from paginated reads—such as when synchronizing an aggregate component with a source-of-truth table—you must manage concurrency control to avoid Optimistic Concurrency Control (OCC) conflicts and performance issues [4][5]. Because aggregates often use internal B-tree structures where operations on adjacent keys or aggregate nodes can interfere with one another, high-frequency writes or large-scale re-indexing can lead to contention [4][6]. Recommended practices for rebuilding or updating aggregates include: 1. Use Idempotent Operations: Use methods such as insertIfDoesNotExist, replaceOrInsert, or deleteIfExists rather than direct inserts or replacements [5][7]. This allows the aggregate state to reach consistency even if the migration or update process is interrupted or re-run [5][7]. 2. Batching and Pagination: When processing large datasets, use pagination to handle data in smaller, discrete batches rather than loading entire structures into memory [6]. 3. Minimized Contention: Be aware that reading an entire aggregate structure creates a broad read dependency that causes the operation to conflict with any mutation modifying that structure [4][5]. Use bounds to restrict the scope of your reads and writes wherever possible [6]. 4. Migration Patterns: For initial backfills, use a background migration that processes data in paginated batches, and ensure that live traffic writes are handled idempotently to converge on the correct state [5][7].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- analytics.ts outline ---'
ast-grep outline packages/backend/convex/analytics.ts --view compact || true

printf '%s\n' '--- target sections ---'
sed -n '300,450p' packages/backend/convex/analytics.ts

printf '%s\n' '--- related symbols and call sites ---'
rg -n -C 4 'replaceSelectionDistributions|rebuildSelectionDistributions|analyticsStats|totalProjects|quarantine' packages/backend/convex

Repository: AmanVarshney01/create-better-t-stack

Length of output: 25525


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- ingestion and repair mutations ---'
sed -n '1,125p' packages/backend/convex/analytics.ts
sed -n '430,490p' packages/backend/convex/analytics.ts

printf '%s\n' '--- helper update semantics ---'
sed -n '1,245p' packages/backend/convex/analytics_helpers.ts

printf '%s\n' '--- schema fields ---'
sed -n '35,65p' packages/backend/convex/schema.ts

printf '%s\n' '--- scheduled or external callers ---'
rg -n -C 5 'rebuildSelectionDistributions|quarantineInvalidEvents|quarantineInvalidEventsBatch|recordAnalytics|internal\.analytics' packages/backend packages --glob '!packages/backend/convex/analytics.ts'

Repository: AmanVarshney01/create-better-t-stack

Length of output: 15670


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
# Read-only model of the active-event set used by the rebuild.
initial = {"A": "frontend-a", "B": "frontend-b"}
scanned = dict(initial)

# Concurrent mutations: ingest C, then quarantine A.
current = dict(initial)
current["C"] = "frontend-c"
del current["A"]

expected_total = len(scanned)
stats_total_after_mutations = len(current)

print({
    "scanned_active_events": scanned,
    "current_active_events": current,
    "expectedTotal": expected_total,
    "stats.totalProjects": stats_total_after_mutations,
    "count_check_passes": expected_total == stats_total_after_mutations,
    "distributions_equal": sorted(scanned.values()) == sorted(current.values()),
})
PY

Repository: AmanVarshney01/create-better-t-stack

Length of output: 409


Protect replacement from concurrent analytics changes.

Separate ctx.runQuery calls and the later ctx.runMutation do not share one snapshot. An insertion followed by quarantine can keep stats.totalProjects equal to expectedTotal, so the check passes while the distributions describe a different active-event set.

Serialize analytics ingestion, quarantine, and rebuild operations. Alternatively, add a monotonic analytics-stats revision, compare it in replaceSelectionDistributions, and retry when it changes.

Comment on lines 76 to +95
const AnalyticsFrontendListSchema = z
.array(FrontendSchema)
.max(FRONTEND_VALUES.length)
.refine(hasUniqueValues)
.refine(hasNoMixedNone);
.refine(hasNoMixedNone)
.transform(normalizeAnalyticsSelection);

const AnalyticsAddonListSchema = z
.array(AnalyticsAddonSchema)
.max(ANALYTICS_ADDON_VALUES.length)
.refine(hasUniqueValues)
.refine(hasNoMixedNone);
.refine(hasNoMixedNone)
.transform(normalizeAnalyticsSelection);

const AnalyticsExampleListSchema = z
.array(ExamplesSchema)
.max(EXAMPLES_VALUES.length)
.refine(hasUniqueValues)
.refine(hasNoMixedNone);
.refine(hasNoMixedNone)
.transform(normalizeAnalyticsSelection);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

For Zod 4.4.3, does z.array(...).transform(fn).optional().parse(undefined)invokefn? Does z.array(...).optional().transform(fn).parse(undefined)invokefn?

💡 Result:

In Zod 4.4.3, both scenarios will invoke the transform function when passed undefined. 1. z.array(...).transform(fn).optional().parse(undefined): The transform function fn is invoked. In Zod 4, $ZodTransform is designed to run even when the input is undefined. To preserve the expected behavior where an outer .optional() results in undefined rather than an error or a transformed value, $ZodTransform attaches a fallback flag to the result [1]. When the outer ZodOptional encounters this fallback flag on a result derived from an undefined input, it short-circuits the process and returns undefined [1]. 2. z.array(...).optional().transform(fn).parse(undefined): The transform function fn is invoked. Because .optional() is applied before the .transform(fn), the input to the transform is undefined. Zod 4 transforms do not short-circuit on undefined by default; they process the undefined input, meaning the function fn will be executed with undefined as its argument [1]. In summary, placing .optional() after .transform() allows Zod to handle the undefined output via its internal fallback mechanism [1], whereas placing .optional() before .transform() means the transform function itself receives and must handle the undefined input [1].

Citations:


🏁 Script executed:

#!/bin/bash
set -eu
printf '%s\n' '--- package files ---'
git ls-files 'packages/types/*' 'packages/types/test/*' | sed -n '1,120p'
printf '%s\n' '--- analytics source ---'
cat -n packages/types/src/analytics.ts | sed -n '1,180p'
printf '%s\n' '--- analytics tests ---'
cat -n packages/types/test/analytics.test.ts | sed -n '1,240p'
printf '%s\n' '--- zod declarations ---'
rg -n '"zod"|zod@|Zod' package.json packages/types package-lock.json pnpm-lock.yaml yarn.lock 2>/dev/null | sed -n '1,160p'

Repository: AmanVarshney01/create-better-t-stack

Length of output: 10599


🏁 Script executed:

#!/bin/bash
set -eu
printf '%s\n' '--- locked Zod version ---'
rg -n -C 3 '(^|/)(zod|`@zod/`)' pnpm-lock.yaml package-lock.json yarn.lock bun.lock bun.lockb 2>/dev/null | sed -n '1,160p' || true
printf '%s\n' '--- available local Zod sources ---'
find . -path '*/node_modules/zod*' -o -path '*/node_modules/@zod*' | sed -n '1,80p'
printf '%s\n' '--- relevant source references ---'
rg -n 'ZodOptional|\$ZodOptional|ZodTransform|\$ZodTransform|fallback' . --glob '!**/node_modules/**' --glob '!**/.git/**' | sed -n '1,160p'

Repository: AmanVarshney01/create-better-t-stack

Length of output: 5673


🌐 Web query:

Zod v4.4.3 source $ZodOptional $ZodTransform undefined parse optional transform

💡 Result:

In Zod v4.4.3, the interaction between $ZodOptional and $ZodTransform is governed by a runtime "fallback" mechanism designed to maintain consistent undefined handling [1][2]. Core Mechanisms: - Fallback Payload: $ZodTransform stamps every invocation result with a fallback: true flag in the internal ParsePayload [1][2][3]. - Short-Circuiting: $ZodOptional checks this flag; when it encounters a fallback result from an inner transform while the original input was undefined, it "clobbers" the transformed output and short-circuits the parse to return undefined [1][2]. - Runtime Optionality: Since v4.4.0+, $ZodTransform (and by extension, z.preprocess) declares optin === "optional" at runtime [1][2]. This allows schemas with user-written input handlers (like transforms or preprocessors) to accept undefined even if the static type suggests otherwise [2]. Key Behaviors: - Preserved Stability: This design ensures that chains like z.preprocess(fn, schema).optional().parse(undefined) correctly return undefined, maintaining compatibility with long-standing Zod behaviors [1][2]. - Transformation and Key Optionality: While undefined input is handled, applying .transform() to a schema in Zod v4 generally changes the resulting type from "key-optional" (e.g., { prop?: T }) to "value-optional" (e.g., { prop: T | undefined }) [4]. This is an intentional design choice; to preserve key optionality in the output, you must explicitly omit the key from the returned object in your transform function [5][4]. Summary of Source Roles: - $ZodTransform: Responsible for the fallback flag, signaling that the output is provisional for undefined inputs [2][3]. - $ZodOptional: Responsible for consuming the fallback flag to ensure that undefined inputs are treated correctly by wrapping schemas [1][2]. - $ZodPreprocess: Structurally pins its optionality (optin) to the inner schema (via lazy metadata), ensuring that optionality placed inside a preprocessor correctly affects the parent object's parsing logic [6].

Citations:


🏁 Script executed:

#!/bin/bash
set -eu
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
curl -fsSL "https://raw.githubusercontent.com/colinhacks/zod/v4.4.3/packages/zod/src/v4/core/schemas.ts" \
  -o "$tmpdir/schemas.ts"
printf '%s\n' '--- exact version source: optional and transform ---'
rg -n -C 18 'function handleOptionalResult|class.*Optional|\\$ZodOptional|\\$ZodTransform|fallback' "$tmpdir/schemas.ts" | sed -n '1,320p'
printf '%s\n' '--- runtime availability ---'
command -v node || true
command -v bun || true
command -v npm || true

Repository: AmanVarshney01/create-better-t-stack

Length of output: 11223


🏁 Script executed:

#!/bin/bash
set -eu
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
curl -fsSL "https://raw.githubusercontent.com/colinhacks/zod/v4.4.3/packages/zod/src/v4/core/schemas.ts" \
  -o "$tmpdir/schemas.ts"
printf '%s\n' '--- optional parser ---'
sed -n '3474,3528p' "$tmpdir/schemas.ts"
printf '%s\n' '--- object property parser ---'
rg -n -C 20 'def.keyType|isOptional|optin|keyResult' "$tmpdir/schemas.ts" | sed -n '1,260p'
printf '%s\n' '--- package entrypoints and runtime files ---'
curl -fsSL "https://registry.npmjs.org/zod/4.4.3" | node -e '
let s=""; process.stdin.on("data",d=>s+=d).on("end",()=> {
  const p=JSON.parse(s);
  console.log(JSON.stringify({version:p.version,main:p.main,module:p.module,types:p.types,files:p.files}, null, 2));
})'

Repository: AmanVarshney01/create-better-t-stack

Length of output: 11741


🏁 Script executed:

#!/bin/bash
set -eu
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
curl -fsSL "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz" | tar -xz -C "$tmpdir"
node - <<'JS' "$tmpdir/package"
const path = process.argv[2];
const { z } = require(path);
const calls = [];
const normalize = (value) => {
  calls.push(value);
  return value && value.length > 0 ? value : ["none"];
};
const transformedThenOptional = z.array(z.string()).transform(normalize).optional();
const optionalThenTransformed = z.array(z.string()).optional().transform(normalize);
const objectOuterOptional = z.object({ field: transformedThenOptional });
const objectInnerOptional = z.object({ field: optionalThenTransformed });

for (const [name, schema] of [
  ["transform().optional()", transformedThenOptional],
  ["optional().transform()", optionalThenTransformed],
  ["object outer optional", objectOuterOptional],
  ["object inner optional", objectInnerOptional],
]) {
  calls.length = 0;
  const result = schema.safeParse(name.startsWith("object") ? {} : undefined);
  console.log(name, JSON.stringify({
    success: result.success,
    data: result.success ? result.data : undefined,
    calls,
  }));
}
JS

Repository: AmanVarshney01/create-better-t-stack

Length of output: 436


Normalize missing selections inside each list schema.

The outer .optional() bypasses normalizeAnalyticsSelection for missing values. Missing frontend, addons, and examples values therefore remain undefined instead of becoming ["none"].

Apply .optional() before .transform(normalizeAnalyticsSelection) in each list schema. Remove the outer .optional() calls and add missing-field assertions to packages/types/test/analytics.test.ts.

This branch was successfully deployed

1 active deployment
Preview — 99093d7d Deployed Aug 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant