Dmitry_Tkachenko/forge

Fork 0

Files

T

Dmytro Tkachenko 9868b18818 Agents

2026-08-29 12:55:39 +03:00

19 KiB

Raw Permalink Blame History

ADR: The Chrome extension as the single sync engine for SNOW + Jira

Status: Proposed (design input for a later build — no code here)
Date: 2026-08-27
Author: architect
Supersedes: nothing — extends the existing SNOW-only sync (extension/, /api/sync)

1. Context

Today the extension syncs ServiceNow only. background.js runs a collector in the rbassist.service-now.com page (world: MAIN), pages the Table API same-origin using the live session cookie + g_ck, maps sc_req_item rows to the FORGE ticket shape, and POSTs them in 100-row chunks to <serverUrl>/api/sync with Authorization: Bearer fg_….

Jira data currently only exists in the seed dump (jira_status_map 127 rows, jira_board_state 1045 rows — see FORGE-parity-spec.md). It goes stale the moment the dump is loaded. The live tickets.jira JSONB is typed as JiraInfo and is only ever written by the seed path, never refreshed. Its declared shape is not 5 fields — it is 6 (server/types.ts:11-18, mirrored in client/src/types/ticket.types.ts:9-16):

interface JiraInfo {
  status?; statusChangedAt?; key?; url?; assignee?;   // the 5 populated by seed today
  movements?: { at: string; who: string }[];          // ALREADY declared — see §5
}

The seed only populates the first five; movements exists in the type but is currently unwritten. This matters: movements is not a field we get to invent (§5).

We want one "Sync now" to refresh both sources so the Active tab (chart 17 Jira status durations, chart 19/20 movements) and the future Jira board page stay live.

Load-bearing finding — the current upsert cannot be reused for Jira as-is. server/db.ts upsertTickets ON CONFLICT does status=EXCLUDED.status, state=EXCLUDED.state, short_desc=EXCLUDED.short_desc, assigned_to=EXCLUDED.assigned_to, assignment_group=EXCLUDED.assignment_group, last_activity_at/by=EXCLUDED.…, updated_at=EXCLUDED.… — these are overwritten, not COALESCE-preserved. A Jira-only payload routed through /api/sync would pass normalizeIncoming, which defaults status→'active', state→'', shortDesc→'', assignedTo→null, assignmentGroup→null, last_activity_*→null — wiping the SNOW core fields of every matched ticket (and mis-flipping closed RITMs back to active). Only jira itself is COALESCE-merged. So Jira must not ride the same endpoint/upsert.

2. Decision

Keep SNOW exactly as-is (in-page same-origin session collector → /api/sync).
Add Jira as a second transport in the same extension, using direct Jira Cloud REST from the service worker authenticated with a Jira API token (email + token, HTTP Basic). No Jira browser tab required.
Route Jira through a new, dedicated server endpoint /api/sync/jira that performs an attach-only UPDATE — it writes only jira (JSONB) and jira_key, keyed by RITM number, and never touches status/state/assignee/activity. This sidesteps the clobber above and gives clean partial-failure semantics.
No DB shape change. Jira status/durations/movements go inside the existing jira JSONB, which the extension sends as one enriched object. Additive only.
"Sync now" = SNOW first, then Jira (sequential, so Jira attaches to freshly-synced rows), each phase chunked and reporting its own count.

Why the SNOW-session / Jira-API-key split (not one mechanism)

	ServiceNow	Jira Cloud
Auth we have	Live browser session cookie + `g_ck`	First-class API token (Atlassian id.atlassian.com → API tokens)
Personal API token	Not reliably available / instance-policy dependent; storing SNOW creds is worse	Designed for exactly this; scoped, revocable
Needs a logged-in tab	Yes (already the case; user is in SNOW all day)	No — SW fetch with host permission works headless
CSRF	`X-UserToken: g_ck` required	Not applicable (Basic auth)

The split is the cheap option on both sides: SNOW keeps the zero-secret session approach that already works; Jira uses the mechanism Atlassian actually blesses. Forcing symmetry (e.g. scraping a Jira tab same-origin) would add a fragile MAIN-world collector and require the user to keep a Jira tab open — strictly worse than a token.

Does Jira require a board id? Yes — and support a list, default one.

To reproduce the original board (rapidView=13793) and its per-status durations + movements, the extension reads the Agile REST endpoint GET /rest/agile/1.0/board/{boardId}/issue (issues in board order) plus each issue's changelog (GET /rest/api/3/issue/{key}?expand=changelog) to reconstruct status transition timestamps → durations/movements. Board order and column mapping only exist per board, so a board id is mandatory for board-faithful output. A pure JQL search (/rest/api/3/search) does not need a board but loses column order and the board's status→column mapping.

Recommendation: primary input is one board id (the 13793 analogue). Store it as a list so a second board can be added later without a settings migration, but the UI defaults to a single field. Provide an optional JQL override for power cases (e.g. project = XYZ AND updated >= -14d); when JQL is set it augments the board fetch's filter, it does not replace the board (we still need the board for column mapping).

3. Settings schema (extension options → `chrome.storage.local`)

{
  // FORGE (unchanged names — back-compat with today's build)
  serverUrl:        "https://forge.mycloud.dp.ua",   // FORGE API domain
  token:            "fg_…",                            // FORGE portal key, minted at Admin→Tokens, revocable

  // Jira (new)
  jira: {
    baseUrl:        "https://rocketmill.atlassian.net", // Jira Cloud site
    email:          "svc-forge@…",                       // Atlassian account email (Basic auth username)
    apiToken:       "ATATT…",                            // Jira API token (Basic auth password) — SECRET
    boardIds:       [13793],                             // list; UI defaults to one
    jql:            ""                                   // optional override/filter, may be blank
    closedLookbackDays: 14                               // 0 = active-only (see §6)
  }
}

FORGE Basic-of-nothing: FORGE keeps Authorization: Bearer <fg_ token>.
Jira auth header: Authorization: Basic base64(email + ":" + apiToken).
On Save, request host permission for both origins (as options.js already does for the FORGE origin): the FORGE server origin and https://<site>.atlassian.net/*.
A "Test Jira" button (mirror of the existing "Test") calls GET {baseUrl}/rest/api/3/myself and reports 200/401.

4. Two transports & manifest implications

SNOW: unchanged. host_permissions: ["https://rbassist.service-now.com/*"] stays required; collector runs in the page; session cookie + g_ck do the work.
Jira: fetched from the service worker (not a page). In MV3, a service-worker fetch to a host listed in host_permissions is exempt from page CORS — the extension is treated as a first-party origin for granted hosts, so Atlassian's (restrictive) CORS headers are irrelevant. This only holds with the host permission granted; without it the fetch is a normal cross-origin call and fails preflight.
Add the Jira site to optional_host_permissions and request it dynamically at Save time (same pattern as the FORGE origin today), rather than hard-coding a static host_permissions entry — the site host is per-deployment and least-privilege favors granting exactly the one instance the user configures. optional_host_permissions already contains https://*/*, which technically covers it, but an explicit narrow grant is cleaner and survives a future tightening of that wildcard.
No new manifest permissions needed (storage, scripting already present; Jira uses neither scripting nor tabs).

5. Dedup & merge

FORGE key is number (RITM). No ticket dupes — that invariant is untouched; Jira never inserts a ticket.
RITM ↔ Jira link: for each Jira issue, resolve the RITM number from customfield_26001 (holds the RITM), falling back to a summary regex (/RITM\d+/), exactly as the initial app did. Build a map RITM number → enriched jira object.
Jira issue with no RITM: skip it and count it. FORGE is RITM-centric and single-user; an unlinked Jira issue has nowhere to attach and creating a ghost ticket would violate the "no dupes / SNOW owns the ticket row" model. Report the unlinked count in the sync status so the user knows a link (customfield/summary) is missing. (A future "orphan Jira" store is out of scope — not kanban, not this ADR.)

What lands in jira JSONB (superset of today's shape):

{ key, url, status, statusChangedAt, assignee,     // 5 existing fields, unchanged
  movements,         // EXISTING field — reuse shape [{ at, who }] (NOT { from, to, at })
  statusDurations,   // NEW additive — { "In Progress": mins, "In UAT": mins, ... }
  board }            // NEW additive — { id, column } for the board page

movements already exists in the JiraInfo type (server/types.ts:17 + client/src/types/ticket.types.ts:15) as { at: string; who: string }[]. The extension must populate that existing shape, not redefine it to { from, to, at }. Reconstruct who from the changelog author and at from the transition timestamp. (I confirmed movements currently has no runtime consumer — only the two type declarations — so a different shape could be adopted, but doing so is a deliberate change to the JiraInfo contract in two TS files, not "additive JSONB." Recommendation: keep { at, who }. If the chart genuinely needs from/to, add them as extra optional keys on each entry ({ at, who, from?, to? }) rather than dropping at/who — that stays backward-compatible and is still a one-line JiraInfo edit, called out here so the engineer expects it.)
statusDurations and board are genuinely new, additive optional keys — add them to the JiraInfo interface (both files) alongside movements. This is a type-declaration touch, not a DB shape change: the jira column is already JSONB and stores whatever the object holds. Flagging it explicitly so it isn't mistaken for a zero-code change.
Server (DB) shape change: none to columns. The jira column is already JSONB. The new endpoint replaces the whole object per ticket (the extension always sends the complete enriched object it just computed), so there is no partial-merge ambiguity and no reshape of any existing column. jira_key (existing TEXT column) is set from jira.key when present. The only code-level shape edit is the additive JiraInfo TS interface above.

New endpoint contract (attach-only)

POST /api/sync/jira        (requireToken — same fg_ Bearer as /api/sync)
body: { issues: [ { number, jira: {…enriched…} }, … ] }   // chunked, 100
per row:  UPDATE tickets
          SET jira = $2::jsonb,
              jira_key = COALESCE($3, jira_key),
              synced_at = NOW()
          WHERE number = $1
resp: { updated: <rows hit>, unmatched: <numbers not found>, unlinked: <issues w/o RITM> }

Because it is an UPDATE … WHERE number =, a Jira payload for an RITM not yet in FORGE simply affects 0 rows (counted as unmatched) — it cannot create a stub row or flip status/state/assignee. This is the whole reason for a separate endpoint rather than folding into /api/sync.

6. Sync flow & scheduling

Sync now (popup) → service worker runSync():

Phase A — SNOW (unchanged): collect active RITMs same-origin → POST /api/sync in 100-chunks. On failure: abort before Phase B (don't attach Jira to a stale ticket set) and report the SNOW error as today.
Phase B — Jira: for each boardId, fetch board issues (+ changelog), resolve RITM, build enriched jira objects, → POST /api/sync/jira in 100-chunks. On failure: Phase A is already committed and intact; surface a warning ("SNOW synced ✓, Jira failed: …") rather than a hard error. Report updated / unmatched / unlinked.
Push a combined { state, snowCount, jiraUpdated, jiraUnmatched, at } to syncStatus for the popup.

Sequential, not parallel: Jira must attach to rows SNOW just wrote.
Chunking: 100 on both push directions (matches today). Jira read is paged by the Agile API (maxResults/startAt, 50–100) — page defensively with a hard ceiling like the SNOW collector's offset < 2000 guard.
Separate endpoint, not folded: decided in §2/§5 — clobber-safety + independent partial-failure reporting.
Scheduling: keep manual "Sync now" for v1 (single-user, user is at the desk). A chrome.alarms periodic sync is a trivial later add but out of scope; note that periodic Jira sync consumes API-token rate budget even when idle.

7. Scope of tickets

SNOW: stays active-only (the collector query is active=true).
Jira: default active board + a small closed lookback (closedLookbackDays, e.g. 14) so recently closed issues' durations/movements stay fresh for charts 17/19/20. 0 = active-only.
Not full history. Trade-offs:
- For historical sync: analytics freshness on old tickets.
- Against (decisive): Jira Cloud API-token rate limits + wall-clock cost of walking every issue's changelog; and history is already owned by the seed dump (analytics_data.ticketsMeta 954 rows is THE historical dataset). Live-syncing 1000+ closed issues on every "Sync now" is wasteful and slow.
- Verdict: seed handles history; the extension keeps active + a short closed window. If someone needs a full historical refresh, that's a re-seed, not a per-click sync.

8. Security

Jira API token is a real secret in chrome.storage.local — which is not encrypted at rest and is readable by anyone with the OS user's Chrome profile on disk. Mitigations to bake into the build:
- Use a dedicated low-privilege Jira service account with read-only project access, not a personal admin token. Blast radius on leak = read a board.
- Store email + API token, never a password. API tokens are individually revocable from Atlassian without disturbing the account.
- Never log the token, the Authorization header, or issue bodies (the SNOW collector already treats g_ck/token this way — hold the same line for Jira).
- Document in options UI that the token is stored locally and to revoke it from Atlassian if the machine is compromised.
FORGE fg_ token is already revocable via Admin→Tokens and requireToken-gated — rotate freely; the new /api/sync/jira reuses the same Bearer, no new server secret.
CORS / host model: covered in §4 — Jira REST works because the SW holds the Jira host permission (CORS-exempt for granted hosts); grant exactly the one Atlassian site, least-privilege, requested at Save.
Server input hardening: /api/sync/jira must validate number is present and jira is an object, and (like sanitizeActivity) coerce the enriched sub-fields before writing JSONB, so a malformed movements/statusDurations can't later crash a render.

9. Alternatives considered

A. Fold Jira into /api/sync. Rejected — the ON CONFLICT overwrites SNOW core fields from a thin Jira payload (§1). Would require rewriting the upsert to COALESCE status/ state/assignee, which then breaks SNOW's own need to set those. A second endpoint is simpler and safer than making one upsert serve two very different payloads.
B. Scrape Jira same-origin from a Jira tab (mirror SNOW). Rejected — needs a logged-in Jira tab, a fragile MAIN-world collector, and gives no advantage over the sanctioned API token.
C. Server-side Jira sync (cron on the FORGE box, no extension). Rejected for now — it moves the Jira secret to the server (fine) but splits sync into two engines, contradicting the goal of one "Sync now", and the RITM↔Jira resolution logic would live in two places. Revisit only if we later want unattended periodic sync.
D. New jira_* columns / a jira_status table. Rejected — violates "prefer additive JSONB, no existing-column reshape"; the jira JSONB already exists and is the right home.
E. JQL-only, no board id. Rejected as the default — loses column order/mapping the board page needs; kept as an optional filter override.

10. Risks

Jira changelog walking is the expensive part; a large board × per-issue changelog fetch can be slow / hit rate limits. Mitigate with the closed-lookback window and paging ceilings.
customfield_26001 id is instance-specific — if it differs on this Jira site the RITM-link falls back to summary regex only; surface the unlinked count so this is visible.
optional_host_permissions wildcard (https://*/*) is broad; the explicit Atlassian grant is cleaner but the wildcard's presence is a standing review flag (unrelated to this change, worth noting to /security).
Token in chrome.storage.local (§8) — accepted risk, mitigated by service-account scope.

11. Migration

DB: none. No column add, no reshape — jira JSONB and jira_key already exist. initDB() is untouched; no manual SQL script, no rollback needed.
Types: additive — extend the existing JiraInfo interface (server/types.ts + client/src/types/ticket.types.ts) with new optional keys statusDurations and board; reuse the already-declared movements?: { at, who }[] (do not redefine it). Optional keys keep every existing consumer compiling.
Server: purely additive — new POST /api/sync/jira handler + a small attachJira(number, jira) in server/db.ts. Old clients that only hit /api/sync keep working unchanged.
Extension: additive options fields + Phase B. An un-upgraded extension still syncs SNOW.
Rollout order: ship the server endpoint first (inert until called) → then the extension update → then populate the Jira settings. Fully backward-compatible at every step.

engineer — build the additive POST /api/sync/jira + attachJira() (attach-only UPDATE, §5) first; then extend the extension options (§3 schema) and background.js Phase B (§6). Route the DB touch through /dba and the Jira-token storage through /security before merge.

19 KiB Raw Permalink Blame History Unescape Escape