# XP Isolation Research — SimTrade vs Learning

**Research only — no code changes.** Scope: determine whether SimTrade XP and Learning XP are shared or isolated, and what it would take to separate them on the leaderboards.

---

## TL;DR

- There is **one global XP ledger** (`user_xp.total_xp` + `xp_events`) shared by Learning, SimTrade, Practice Mode, and any client-reported events.
- The **SimTrade leaderboard is already isolated** — it's ranked by `return_pct` via `sim_leaderboard_cache`, not by XP. So SimTrade → Learn pollution exists only in one direction: SimTrade XP leaks into the **Learn / XP leaderboards**.
- The **Learn leaderboards** (`GET /api/learn/leaderboard`, `GET /api/xp/leaderboard`) both read `user_xp.total_xp`, which is a single running counter with no feature attribution. A user who grinded only SimTrade milestones still shows up on the Learn leaderboard.
- There is **no `feature`/`source` column** on `xp_events`. The only way to distinguish SimTrade vs Learn XP today is by parsing the `action` string (SimTrade actions are prefixed `simtrade_*`).

---

## 1. Current XP Architecture

### Tables (`supabase/migrations/20260401000000_xp_system_tables.sql`)

- **`user_xp`** — one row per user. Columns: `user_id, total_xp, current_streak, best_streak, last_active_date, streak_frozen_count, updated_at`. **Single `total_xp` counter, no feature breakdown.**
- **`xp_events`** — append-only ledger. Columns: `id, user_id, action, xp_earned, reference_id, created_at`. Unique index on `(user_id, action, reference_id)` for idempotency. **No `feature` / `source` / `category` column.**
- **`daily_quests`** — per-user per-day quest progress (Learn-specific: lesson/quiz/research quests).
- **`streak_history`** — daily activity calendar.
- **`sim_milestones`** (separate SimTrade-only table) — milestone ledger for SimTrade; stores `xp_awarded` inside `metadata` JSON as the source of truth for the "XP earned from SimTrade" figure shown on the performance screen.
- **`sim_leaderboard_cache`** (separate SimTrade-only table) — pre-ranked daily cache for the SimTrade leaderboard, ordered by `return_pct` (not XP).

### Core service module

`services/main/app/services/xp_service.py`:
- `_process_xp_award(user_id, action, xp_amount, reference_id)` is the single chokepoint: inserts into `xp_events`, upserts `user_xp.total_xp`, advances streak, updates daily quest progress. All callers funnel through here.
- `record_xp_event(action, reference_id)` — looks `xp_amount` up from `xp_repository.XP_REWARDS` (used by the client POST endpoint).
- `award_xp_internal(action, xp_amount, reference_id)` — caller supplies `xp_amount` directly (used by Learn, Practice, SimTrade server-side code).
- `get_leaderboard(...)` → `xp_repository.get_leaderboard()` → `SELECT user_id, total_xp FROM user_xp ORDER BY total_xp DESC`.
- `get_combined_leaderboard(...)` → fetches `get_leaderboard`, `get_friends_leaderboard`, `get_teams_leaderboard` concurrently. All three query `user_xp.total_xp`.

---

## 2. XP Touchpoints Inventory

Every place that awards XP (grepped `xp_service|award_xp|record_xp_event` across `app/`):

| Caller | Actions it writes | Notes |
|---|---|---|
| `services/learning_service.py` (`_award_xp`) | `lesson_complete`, `quiz_pass`, and lesson/module-level awards with amounts from CMS content | Used across lesson completion, quiz pass, etc. (NAN-65/NAN-186) |
| `services/practice_service.py` | `practice_complete` | Practice Mode sessions (NAN-298) |
| `services/simtrade_performance_service.py` (`_award_milestone_xp`) | `simtrade_first_trade`, `simtrade_first_profit`, `simtrade_beat_sp500_monthly`, `simtrade_beat_sp500_all_time`, `simtrade_positive_week`, `simtrade_positive_month`, `simtrade_ten_pct_return`, `simtrade_fifty_pct_return` | **All 8 SimTrade milestones award global XP.** `reference_id = milestone_key`. Also persisted in `sim_milestones.metadata.xp_awarded`. |
| `services/xp_service.py::_update_quest_progress` | `daily_quest_lesson_complete`, `daily_quest_quiz_pass`, `daily_quest_research_time`, `daily_quest_bonus` | Learn-only daily-quest rewards |
| `routers/xp.py POST /api/xp/events` | Any action in `_ALLOWED_CLIENT_ACTIONS` = `{lesson_complete, quiz_pass, watchlist_add, earnings_read, ask_anything_first, daily_login, research_time}` | Client can push these. All are Learn/research-flavored — SimTrade actions are **not** in this allowlist (SimTrade XP only flows via server-side milestone evaluation). |

### Rewards table (`xp_repository.XP_REWARDS`)

```
lesson_complete: 50
quiz_pass: 75
watchlist_add: 25
simtrade_first: 100       ← legacy; not written by current SimTrade code
earnings_read: 30
ask_anything_first: 25
daily_login: 10
beat_sp500: 500           ← legacy; not written by current SimTrade code
win_competition: 1000     ← referenced nowhere in live code paths I grepped
```

Note the naming split: the rewards map still has old short names (`simtrade_first`, `beat_sp500`), but `simtrade_performance_service` writes the **new prefixed names** (`simtrade_first_trade`, `simtrade_beat_sp500_monthly`, etc.) with amounts sourced from `_MILESTONE_DEFS` — so the stale entries in `XP_REWARDS` are effectively dead.

---

## 3. Leaderboard Scoping

| Endpoint | Source | Ordered by | Isolation status |
|---|---|---|---|
| `GET /api/xp/leaderboard` | `user_xp.total_xp` | XP desc | **Mixed** — includes SimTrade XP |
| `GET /api/learn/leaderboard` (combined: global + friends + teams) | `user_xp.total_xp` | XP desc | **Mixed** — includes SimTrade XP |
| `GET /api/learn/teams/{id}/leaderboard` | `user_xp` joined via team membership | XP desc | **Mixed** — includes SimTrade XP |
| `GET /api/simtrade/leaderboard` | `sim_leaderboard_cache` | `return_pct` desc | **Already isolated** — no XP involved |
| `GET /api/simtrade/competitions/{id}/leaderboard` | competition ranker | competition scoring | Isolated from XP |

### Performance screen
`GET /api/simtrade/performance` computes its own `xp` figure by summing `sim_milestones.metadata.xp_awarded` (not reading from `user_xp` or `xp_events`). So the "SimTrade XP" number shown to users **is** already source-of-truth separate from `total_xp`. The two numbers will drift only if `_award_milestone_xp` fails on one leg — which is handled via `xp_pending` retry.

---

## 4. Isolation Gap Analysis

**What's already isolated:**
- SimTrade leaderboard (different table, different metric).
- SimTrade performance screen's "XP" total (reads `sim_milestones` directly).
- Daily-quest progress (Learn-specific — only lesson/quiz/research actions map to quest fields).

**What's NOT isolated:**
1. `user_xp.total_xp` is one number. When a user earns `simtrade_first_trade` (+100), their `total_xp` increases and they climb the Learn leaderboard.
2. `xp_service.get_leaderboard` / `get_combined_leaderboard` / `get_friends_leaderboard` / `get_teams_leaderboard` all sort by `user_xp.total_xp`, so Learn/global/friends/teams leaderboards all visibly include SimTrade XP.
3. **User streak + daily activity** is shared: a SimTrade milestone counts as "activity today" and advances the streak counter that the Learn UI shows. This may or may not be desired — the user's concern was about leaderboards, but streaks have the same coupling.
4. The XP `total_xp` → `compute_level()` "beginner/intermediate/advanced/expert/master" rank displayed in Learn also reflects SimTrade XP.

**What would need to change to fully isolate:**
- Either (a) store XP per-feature and expose a scoped total, or (b) exclude SimTrade actions at query time when building the Learn leaderboards.

---

## 5. Recommended Approach (options, not code)

### Option A — Cheapest read path: derived column on `user_xp`
Add two new counters on `user_xp`: `learn_xp` and `simtrade_xp`. Keep `total_xp` as the sum (for "level" badges that users expect to grow from all activity).
- In `_process_xp_award`, route the increment to the right column based on `action.startswith("simtrade_")`.
- Change Learn/global/friends/teams leaderboard queries to `ORDER BY learn_xp DESC`.
- SimTrade leaderboard stays on `return_pct` (unchanged).
- Backfill: one-shot migration summing `xp_events` grouped by `action LIKE 'simtrade_%'` per user.
- **Pros:** single-row read, no query rewrite for the leaderboard hot path. Keeps `total_xp` as a "lifetime all-activity" figure if the UI wants it.
- **Cons:** schema change; need to keep three counters consistent on every award.

### Option B — Feature column on `xp_events` + aggregation view
Add `feature text not null` to `xp_events` (values: `learn`, `simtrade`, `practice`). Create a DB view or materialized view `user_xp_by_feature` that `SUM(xp_earned) GROUP BY user_id, feature`. Leaderboard queries read the view filtered to `feature='learn'`.
- **Pros:** richer breakdown (could show "XP by feature" chart later). Single source of truth.
- **Cons:** every leaderboard query now aggregates the ledger (or hits a materialized view with refresh lag). Slower than Option A for the hot path.

### Option C — Query-time exclusion only
Leave the schema alone. Change `xp_repository.get_leaderboard` to compute `total_xp - (SUM of xp_events where action LIKE 'simtrade_%')` per user. Could be done via a SQL view or a Postgres function.
- **Pros:** no schema change; zero backfill.
- **Cons:** per-query ledger scan for every leaderboard read. Friends/teams variants get expensive. Fragile — relies on the `simtrade_` action prefix being the only discriminator forever.

### Option D — Keep XP global, isolate UI only
Decide that "XP is global across the app" is a product-level truth, and the only fix is removing SimTrade from the Learn leaderboard UI visually — e.g. relabel `/api/learn/leaderboard` to `/api/app/leaderboard` or add a chip that says "global XP". Cheap but doesn't match the user's stated intent ("SimTrade XP should NOT appear on Learning leaderboards").

### Recommendation
**Option A** is the best fit for the stated requirement:
- Minimal query-path change (one-column reorder on an already-indexed table).
- Matches how `action` prefixes are already the de-facto discriminator.
- Keeps `total_xp` as a user-facing "lifetime" metric so existing level/badge logic stays meaningful.
- Also cleanly answers the symmetric question: if we ever build a SimTrade XP leaderboard later, we have the counter ready.

**Open decisions for the user before implementing:**
1. Should the user's **streak** and **daily quest** progress also be Learn-scoped, or does SimTrade activity still count toward "active today"? (Current: SimTrade milestones advance the streak.)
2. Should the level badge (`beginner → master`) be driven by `learn_xp` or `total_xp`?
3. Practice Mode (`practice_complete`) — does it belong to Learn or is it its own bucket?
4. Client-recorded actions `watchlist_add`, `earnings_read`, `ask_anything_first`, `daily_login`, `research_time` — are these "Learn" or "Research"? Today they all flow into the single pot.

---

## Appendix: Key file references

- `supabase/migrations/20260401000000_xp_system_tables.sql` — `user_xp`, `xp_events`, `daily_quests`, `streak_history` DDL
- `services/main/app/services/xp_service.py:144` — `_process_xp_award` chokepoint
- `services/main/app/services/xp_service.py:467` — `get_leaderboard`
- `services/main/app/services/xp_service.py:492` — `get_combined_leaderboard`
- `services/main/app/repositories/xp_repository.py:20` — `XP_REWARDS` map
- `services/main/app/repositories/xp_repository.py:235` — `get_leaderboard` SQL
- `services/main/app/services/simtrade_performance_service.py:25` — `_MILESTONE_DEFS` (8 SimTrade milestones + XP amounts)
- `services/main/app/services/simtrade_performance_service.py:450` — `_award_milestone_xp` → calls `xp_service.award_xp_internal` with `action="simtrade_{key}"`
- `services/main/app/services/simtrade_performance_service.py:985` — performance endpoint reads XP from `sim_milestones.metadata.xp_awarded` (isolated from `user_xp`)
- `services/main/app/services/simtrade_performance_service.py:1009` — SimTrade `get_leaderboard` reads `sim_leaderboard_cache` by `return_pct`
- `services/main/app/routers/xp.py:38` — `_ALLOWED_CLIENT_ACTIONS` (no SimTrade actions)
- `services/main/app/routers/learn.py:602` — `/api/learn/leaderboard` → `xp_service.get_combined_leaderboard`
- `services/main/app/routers/xp.py:236` — `/api/xp/leaderboard` → `xp_service.get_leaderboard`
- `services/main/app/routers/simtrade.py:775` — `/api/simtrade/leaderboard` → `simtrade_performance_service.get_leaderboard`

---

# ADDENDUM: Level Badge System

Follow-up investigation focused on the XP-derived "level" tier (beginner → master) and its relationship to SimTrade milestone badges.

## 1. Where level is computed

`services/main/app/services/xp_service.py:22-37`:

```python
_LEVELS = [
    (0, "beginner"),
    (500, "intermediate"),
    (2000, "advanced"),
    (5000, "expert"),
    (10000, "master"),
]

def compute_level(total_xp: int) -> str:
    level = "beginner"
    for threshold, name in _LEVELS:
        if total_xp >= threshold:
            level = name
    return level
```

Called from exactly one place: `get_xp_stats` (xp_service.py:66) which returns `{ total_xp, current_streak, best_streak, streak_frozen_count, level, today_xp }`.

**Thresholds:**
| XP | Tier |
|---|---|
| 0–499 | beginner |
| 500–1999 | intermediate |
| 2000–4999 | advanced |
| 5000–9999 | expert |
| 10000+ | master |

## 2. What it reads from

`compute_level()` takes a single integer — `user_xp.total_xp`. So today it reflects **all XP** from Learn, SimTrade milestones, Practice, and daily quests. No per-feature breakdown.

## 3. API endpoints that return `level`

- **`GET /api/xp/stats`** — returns `level` on `XpStatsResponse` (`models/xp.py:21`). **Only endpoint that exposes it.**
- `GET /api/simtrade/performance` — does NOT return `level`. It returns `{snapshots, milestones, xp, xp_message}` computed from `sim_milestones.metadata.xp_awarded`.
- `GET /api/xp/leaderboard`, `GET /api/learn/leaderboard` — leaderboard entries don't carry `level`, only `total_xp`.
- `services/learning_service.py:607,661` — `get_xp_stats` is also called to enrich the `POST /api/learn/lessons/{id}/complete` response, but only `total_xp` and `current_streak` are pulled from the dict — **`level` is dropped**.

## 4. Mobile usage

Grepped `/Users/fathoni/Documents/Project/BlockDev/nano-street/mobile/src` for `xp/stats`, `XpStats`, `level`, `compute_level`:

- **`NanoStreetXpAdapter.ts`** exposes only `/api/xp/daily-quest`, `/api/xp/streak/calendar`, `/api/xp/history`. **The mobile app does not call `/api/xp/stats` at all.** The `XpApi` contract has no `fetchXpStats` method.
- **`xpDto.ts`** has no `level` field anywhere — no zod schema consumes it.
- The `"beginner" | "intermediate" | "advanced"` string literals found in mobile are all from the **unrelated `KnowledgeLevelSelector`** (onboarding self-reported experience level, stored as `profiles.experience_level`) and from `OnboardingCompletionScreen`. These are driven by user selection, not by XP.
- **`StatsBar.tsx:36`** has a copy line "Your XP shows how much you've learned. More XP unlocks new levels and features." — but it's tooltip text, not a rendered tier label. The bar itself displays XP number + streak, not a tier.
- No screen renders a `beginner/intermediate/advanced/expert/master` badge from XP.

**Conclusion:** The XP `level` field is currently **dead code on the client side**. It's computed server-side and returned by `GET /api/xp/stats`, but no mobile consumer reads it. Changing or removing it has zero user-visible impact today.

## 5. Relationship to SimTrade `sim_milestones` badges

Completely separate systems:

| Aspect | XP `level` (xp_service) | SimTrade milestones (sim_milestones) |
|---|---|---|
| Table | `user_xp` | `sim_milestones` |
| Values | 5 fixed tiers (beginner/intermediate/advanced/expert/master) | 8 milestone keys (`first_trade`, `first_profit`, `beat_sp500_monthly`, `beat_sp500_all_time`, `positive_week`, `positive_month`, `ten_pct_return`, `fifty_pct_return`) |
| Computed from | Sum of `total_xp` ≥ threshold | SimTrade P&L conditions evaluated in `_evaluate_milestones` |
| Endpoint | `GET /api/xp/stats` → `level` | `GET /api/simtrade/performance` → `milestones[]` |
| Purpose | User progression tier | Achievement trophies |

They overlap only indirectly: earning a SimTrade milestone awards XP which bumps `total_xp`, which may bump `level`. But the milestone names (`first_trade`, etc.) are never mapped to the tier names (`beginner`, etc.).

Note: `_MILESTONE_DEFS` also does **not** reference `trophy_silver` — the task prompt mentioned that name, but current code has 8 achievement keys named after the condition, not trophy metal. No gold/silver/bronze scheme exists in the backend.

## 6. Impact of changing `level` source

### If we switch `level` to `learn_xp` (Option A from main report)
- **Behavior change:** A user who grinds SimTrade milestones (+100 first_trade, +500 beat_sp500_monthly, +500 beat_sp500_all_time, etc.) would no longer climb the XP tier ladder from those activities.
- **User-visible impact today:** **None.** The mobile app does not render `level` anywhere. The only consumers are the `XpStatsResponse` schema and a copy string.
- **Future-facing impact:** If a tier badge is added to the UI later, it will only reflect Learn engagement — aligning with the stated product intent.
- **Backfill:** If `learn_xp` is added via a one-shot migration (sum of non-`simtrade_%` actions), existing users' tiers will drop. E.g., a user at `total_xp = 2100` (2000 from Learn, 100 from `simtrade_first_trade`) currently sits in "advanced"; after switch they'd be at `learn_xp = 2000` → still "advanced". A user at `total_xp = 1100` from pure SimTrade activity currently shows "intermediate"; after switch `learn_xp = 0` → "beginner".

### If we keep `level` on `total_xp`
- `level` remains a "lifetime engagement" metric, unaffected by leaderboard scoping.
- Arguably the more user-friendly default: users aren't punished tier-wise for using multiple features.
- No migration needed for the level field.
- Leaderboard scoping (Option A) can still happen in parallel — add `learn_xp`, point leaderboards at it, **leave `compute_level(total_xp)` unchanged**.

### Recommendation
**Keep `compute_level()` on `total_xp`.** Reasons:
1. No mobile consumer reads it → changing it fixes nothing user-visible.
2. It's the only "you earned XP from everything you did" signal the schema carries, and removing that would hide the fact that SimTrade milestones even exist for users who only check the Learn UI.
3. Decoupling leaderboards from tier badges lets product decide independently whether a future level-badge UI should be Learn-only or cross-feature.

The main-report Option A (add `learn_xp` column, re-point Learn leaderboards to it) is independent of `compute_level` and can proceed without touching the level logic.

## Addendum file references

- `services/main/app/services/xp_service.py:22` — `_LEVELS` thresholds
- `services/main/app/services/xp_service.py:31` — `compute_level()`
- `services/main/app/services/xp_service.py:45` — `get_xp_stats` (sole caller)
- `services/main/app/models/xp.py:21` — `XpStatsResponse.level` field
- `services/main/app/routers/xp.py:57` — `GET /api/xp/stats` endpoint
- `mobile/src/features/learn/service/contracts/XpApi.ts` — mobile XP contract (no `fetchXpStats`)
- `mobile/src/features/learn/service/api/NanoStreetXpAdapter.ts` — mobile adapter (3 XP endpoints, none is `/stats`)
- `mobile/src/features/learn/data/dto/xpDto.ts` — no `level` field
- `mobile/src/features/learn/presentation/components/StatsBar.tsx:36` — only "level" mention in Learn UI (tooltip copy)