📢

Ingestion + Knowledge-Graph Foundation

This version delivers that backbone end-to-end with no real Connector logic: an Org admin can add a Source, an ingestion job runs and flips its status through the lifecycle (proving the pipeline), credentials are stored encrypted, and the Org's data can be exported. Actual crawling of external systems and the reasoning/query layer are explicitly deferred to follow-up work.

NEW

Ingestion + Knowledge-Graph Foundation

This version delivers that backbone end-to-end with no real Connector logic: an Org admin can add a Source, an ingestion job runs and flips its status through the lifecycle (proving the pipeline), credentials are stored encrypted, and the Org's data can be exported. Actual crawling of external systems and the reasoning/query layer are explicitly deferred to follow-up work.

  • Added: Async Org data export — an Org owner/admin can export the Org's full Sources (credentials redacted) and knowledge graph (Nodes, Edges, Evidence) from the Sources page. An Oban worker (unique per Org) builds the JSON, writes it to a private `exports/` S3 prefix, and delivers a short-lived presigned download link via a new `org_data_export_ready` notification (in-app + email)
  • Added: `Galileo.Storage` — a private object-storage seam (SigV4 server-side put with SSE-S3, presigned GET with a content-disposition download filename, plus list/delete), separate from the public-read `FileUploads`
  • Added: `PurgeExpiredExportsWorker` — daily Oban cron that deletes stale `exports/` objects as a safety net beyond the bucket's 7-day lifecycle rule
  • Added: Ingestion — connect a knowledge Source per Org and watch it Sync live. Sources are pull-based (GitHub, docs sites, Notion/Confluence, Jira/Linear, Slack, Datadog) or push-based (signed inbound webhooks), with a `pending → syncing → completed/failed` lifecycle broadcast over PubSub (walking skeleton; per-type Connectors deferred)
  • Added: Encrypted Source credentials at rest via Cloak — API tokens (pull) and signing secrets (webhook) are never stored in plaintext
  • Added: Knowledge-graph data layer — two-family Nodes (Artifact/Concept), directed typed Edges (`from_node → to_node`), and append-only Evidence tying every Node and Edge back to its Source location; every read is Org-scoped and writes are idempotent so re-ingestion converges instead of duplicating