Skip to content

[feat] Identify analytics people by backend user id and track deployments - #8100

Open
mmabrouk wants to merge 2 commits into
release/v0.128.6from
fix/posthog-identify-merge
Open

mmabrouk wants to merge 2 commits into
release/v0.128.6from
fix/posthog-identify-merge

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Goal

Analytics needs one identity per person that the backend decides, not the browser, and one stable id per deployment.

  • One user on two browsers is one person.
  • The same email on two deployments is two people, on self-hosted and on cloud.
  • We can count how many new deployments are created.

Problem

  • The web app identified each browser with a random device id at load, then aliased it to the email after login. PostHog refuses to merge two identified persons, so every browser after the first stayed a separate person.
  • Server events used the email as the person id. The same email on two deployments (for example test@agenta.ai on two self-hosted servers) became one person.
  • There was no deployment id, so new deployments could only be guessed from the first signup.

Change

Identity (API and both web apps, cloud and self-hosted)

  • The person id is the user id from the database. It is unique per deployment.
  • The email and username are person properties only.
  • The browser stays anonymous until the profile loads, then calls identify(user.id, {email, username}). Because the browser is still anonymous, PostHog merges it into that person.
  • Server events (analytics_middleware, user_signed_up_v1, oss_deployment_created) use the user id through one helper, capture_user_event.
  • The device id (generateOrRetrieveDistinctId) and the alias call are removed.
  • identify is sent once per identity change, not on every component mount.

Deployment id

  • Migration oss000000063 creates instances with one row: a random id, created_at, preexisting, announced_at. It runs in both editions and is kept across upgrades because it lives in the database.
  • preexisting is true when the deployment already had users at migration time, so an upgrade is not counted as a new deployment.
  • At startup the API loads the id. Once per deployment (claimed with one UPDATE ... WHERE announced_at IS NULL, so one worker wins), it registers the instance group. For a new deployment it also sends instance_created, without creating a person profile.
  • Every server event carries instance_id and the instance group.
  • GET /profile returns instance_id, and the web apps call posthog.group("instance", id).

Effect on existing data

  • New events go to the user-id person. Persons keyed by email or device id keep their past events. A separate one-off data step can link history where that is safe (on cloud, email to user id per region).
  • Browsers that were already identified with a device id switch to the user id on their next load.

Tests

  • API: new test_analytics_identity.py covers user id as person, email as a property, the instance group, and the middleware path. The signup test checks the user id. The migrations, auth, middlewares and routers unit suites pass: 348 passed, 2 skipped (they need a scratch Postgres).
  • Web: mobile analytics 11/11 and shared analytics 4/4 pass. There are no new tsc errors in oss, mobile or agenta-shared against the base. ESLint and Prettier pass on the changed files.
  • Not run here: the migration against a real Postgres, and a browser check of the merge. Both need a deployed stack.

On cloud, the web app identified each browser with a random device ID
at load, then aliased that ID to the email after login. PostHog refuses
to merge two identified persons, so every browser after the first one
stayed a separate person with the same email.

Cloud now stays anonymous until the profile loads and then identifies
by email, which lets PostHog merge the anonymous browser into the email
person. The alias call is removed. Identification is also sent once per
identity change instead of on every component mount. Self-hosted
behavior is unchanged.
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Summary

Summary by CodeRabbit

  • Analytics
    • Signed-in profiles are now identified by their account ID, with email and username retained as profile details.
    • Visitors remain anonymous until their profile loads; analytics no longer links desktop visitor IDs or aliases profiles by email.
    • Analytics events can now be associated with the deployment they came from.
    • Deployment creation and signup events are attributed to the relevant account when available.
    • Repeated updates avoid re-identifying a profile unless its identity or deployment details change.

Analytics now uses backend user IDs as distinct IDs and deployment instance IDs for group attribution. The API exposes and announces deployment identity, and frontend analytics identifies users after profile data is available.

Changes: Analytics identity and instance attribution

Layer / File(s) Summary
Persist and expose deployment instance identity
api/oss/databases/postgres/migrations/core_oss/versions/*, api/oss/src/utils/instance.py, api/entrypoints/routers.py, api/oss/src/models/api/user_models.py, api/oss/src/routers/user_profile.py
A migration creates an instance row with creation and announcement fields. Startup loads and announces the instance. User profiles include its ID on cached and newly constructed responses.
Capture API events with user and instance identity
api/oss/src/middlewares/analytics.py, api/oss/src/core/auth/supertokens/overrides.py, api/oss/src/services/commoners.py, api/oss/tests/pytest/unit/auth/test_helper.py, api/oss/tests/pytest/unit/middlewares/test_analytics_identity.py
API event capture uses the user ID as the distinct ID and passes email separately. When an instance ID is available, capture adds it to event properties and groups. Signup and deployment event paths use user IDs.
Identify frontend users by backend ID
web/packages/agenta-shared/src/analytics/index.ts, web/packages/agenta-shared/src/types/user.ts, web/packages/agenta-shared/tests/unit/analytics.test.ts, web/mobile/src/features/analytics/*, web/mobile/tests/unit/analytics.test.tsx, web/oss/src/lib/helpers/analytics/*
The shared identity helper returns the backend user ID and optional instance ID. Mobile and OSS clients no longer generate visitor IDs or alias users. They identify after a user ID is available and group users when an instance ID is present.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Startup as API startup
  participant Instance as Instance utilities
  participant Profile as User profile API
  participant Analytics as Web analytics
  participant PostHog
  Startup->>Instance: Load and announce instance ID
  Profile->>Analytics: Provide user ID and instance ID
  Analytics->>PostHog: Identify user and group by instance
Loading
Merge Risk: Low · up to c80e5

The remaining changes are limited to shortening two comments to meet the frontend convention. No established identity or attribution failure prevents merging.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title names the two main changes: analytics people are identified by the backend user ID, and deployments are tracked. The changeset matches both, including the user-ID identity switch across web …
Description check Passed The description is detailed and matches the changeset. It covers the switch from device ID and email aliasing to user-ID identity, the capture_user_event helper, the instances migration with preexisti…
Docstring Coverage Passed Docstring coverage is 81.82% which is sufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentati…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the two main changes: analytics people are identified by the backend user ID, and deployments are tracked. The changeset matches both, including the user-ID identity switch across web and API and the new instances table with instance group and instance_created event. The title is specific and concise, and it avoids file lists and vague terms.

Full details: Description check

Explanation

The description is detailed and matches the changeset. It covers the switch from device ID and email aliasing to user-ID identity, the capture_user_event helper, the instances migration with preexisting and announced_at fields, the instance group and instance_id on GET /profile, and the tests that were added or updated.

Full details: Docstring Coverage

Explanation

Docstring coverage is 81.82% which is sufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 11 functions across 18 files.

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick comments: 1
web/oss/src/lib/helpers/analytics/AgPosthogProvider.tsx: 1

57-59: Trivial · Maintainability & Code Quality · Low value

Shorten the three-line comment to one line.

The frontend rule limits each in-code comment to one short line. It allows longer comments only for a genuinely surprising constraint. This block narrates the cloud identity flow in prose. The same flow is documented in usePostHogAg.ts. Cut it to one line.

♻️ Proposed change
-                        // Cloud stays anonymous until the profile loads, so the later
-                        // identify(email) in usePostHogAg can merge this browser into the
-                        // email person.
+                        // Cloud stays anonymous until usePostHogAg identifies by email.

As per coding guidelines: "At most ONE short line per comment. No multi-line blocks narrating why in prose".

Source: Coding guidelines

Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 4048536 and 7a1574c.

Files selected for processing: 5
  • web/mobile/src/features/analytics/Analytics.tsx
  • web/mobile/src/features/analytics/client.ts
  • web/mobile/tests/unit/analytics.test.tsx
  • web/oss/src/lib/helpers/analytics/AgPosthogProvider.tsx
  • web/oss/src/lib/helpers/analytics/hooks/usePostHogAg.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Railway Preview Environment

Item Value
Status stopped
Commit c80e5d1923bcaa59e190cb2681cc2567fba3d41a
Manual expiry (UTC) Not granted
CI deadline (UTC) Not granted
Workflow https://github.com/Agenta-AI/agenta/actions/runs/41898564530

Dependent tests finished. Post /preview for manual review. Environment absence verified; retained storage requires separate provider accounting.

Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.

…oyment

Analytics now uses one identity that the backend assigns: the user id.
The email is only a person property. The same email on two deployments
is two people, and one user on two browsers is one person. Server
events and the web apps (desktop and mobile, cloud and self-hosted) use
the same id, and the browser stays anonymous until the profile loads.

Each deployment gets a stable instance id, created once by migration
oss000000063 and kept across upgrades. Every event carries it as the
'instance' group and an instance_id property. A new deployment sends
instance_created once; a deployment that already had users when it
upgraded is registered but not counted as new. The profile response
returns the instance id so the web apps can attach it.

The random per-browser device id is removed.
@mmabrouk mmabrouk changed the title [fix] Identify cloud users by email so PostHog merges their devices [feat] Identify analytics people by backend user id and track deployments Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Website preview

Preview URL: https://pr-8100-agenta-website-preview.mahmoud-637.workers.dev

Built from c80e5d1923bcaa59e190cb2681cc2567fba3d41a. This comment updates in place on every push.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick comments: 2
web/packages/agenta-shared/src/analytics/index.ts: 1

25-30: Trivial · Maintainability & Code Quality · Quick win

Shorten the JSDoc block to one line.

The frontend guideline allows at most one short line per comment. This block has six lines and narrates rationale in prose. Keep only the non-obvious why.

♻️ Proposed fix
-/**
- * The analytics person is the user id the backend assigned, the same id the API uses for
- * server events. It is unique per deployment, so one email on two deployments is two people,
- * while one user on two browsers is one person. The email is only a person property.
- * Until the profile loads there is no id and the browser stays anonymous.
- */
+/** Person = backend user id (unique per deployment); email is only a property. */

As per coding guidelines: "Keep in-code comments terse ... At most ONE short line per comment."

Source: Coding guidelines

web/mobile/src/features/analytics/client.ts: 1

37-38: Trivial · Maintainability & Code Quality · Quick win

Shorten this comment to one line.

The frontend guideline allows at most one short line per comment. The comment spans two lines.

♻️ Proposed fix
-                            // The browser stays anonymous until the profile loads; Analytics
-                            // then identifies it by user id.
+                            // Anonymous until `Analytics` identifies by user id.

As per coding guidelines: "Keep in-code comments terse ... At most ONE short line per comment."

Source: Coding guidelines

Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 7a1574c and c80e5d1.

Files selected for processing: 18
  • api/entrypoints/routers.py
  • api/oss/databases/postgres/migrations/core_oss/versions/oss000000063_add_instance.py
  • api/oss/src/core/auth/supertokens/overrides.py
  • api/oss/src/middlewares/analytics.py
  • api/oss/src/models/api/user_models.py
  • api/oss/src/routers/user_profile.py
  • api/oss/src/services/commoners.py
  • api/oss/src/utils/instance.py
  • api/oss/tests/pytest/unit/auth/test_helper.py
  • api/oss/tests/pytest/unit/middlewares/test_analytics_identity.py
  • web/mobile/src/features/analytics/Analytics.tsx
  • web/mobile/src/features/analytics/client.ts
  • web/mobile/tests/unit/analytics.test.tsx
  • web/oss/src/lib/helpers/analytics/AgPosthogProvider.tsx
  • web/oss/src/lib/helpers/analytics/hooks/usePostHogAg.ts
  • web/packages/agenta-shared/src/analytics/index.ts
  • web/packages/agenta-shared/src/types/user.ts
  • web/packages/agenta-shared/tests/unit/analytics.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant