You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf: vectorise the GainLoss decomposition and the CustomerDecisionHierarchy Yule's Q matrix #611
What: two analysis classes do per-row or per-pair Python work that is slow at retail scale. Both predate the current branch.
GainLoss: the six-way decomposition calls process_customer_group once per customer through DataFrame.apply(axis=1). With millions of customers that is millions of Python calls. The same branch logic can be written once with np.select/np.where over the diff columns. process_customer_group is public, so keep it (or deprecate it) and add a test that the vectorised result matches it.
CustomerDecisionHierarchy: the Yule's Q distance matrix is built in a Python loop over every product pair, calling .toarray() on a dense customer vector each time. That is O(products²) iterations over arrays as long as the customer count, so 1,000 products × 1M customers means about 500k iterations over 1M-length arrays. Sparse M @ M.T plus the row sums give the a/b/c/d counts for all pairs in one pass.
Both go against the CLAUDE.md rule to use vectorised operations instead of loops or .apply().
Tests: for each, assert the vectorised output equals the current output on the existing fixtures plus one randomised case.
Activity