Skip to content

0009. Replace igir with an in-cluster Go-based ROM curator

  • Status: accepted
  • Date: 2026-05-25
  • Deciders: lusoris
  • Related: igir suspension commit 0fed9f37, RomM app (apps/media/romm/), cluster Ollama (argocd-apps/ai/ollama.yaml), Open WebUI RAG (apps/ai/cluster-rag-indexer/).

Context

igir was the ROM curator wired between sabnzbd's completed-downloads folder and the RomM library. It is a Node-based tool that loads the entire DAT corpus (No-Intro, Redump, TOSEC, …) into memory before hashing any input files. The repo's working corpus is ~167 DATs at roughly 56 MB on disk; igir's resident heap during a normal run grew past 7 GiB.

Successive mitigations failed:

  1. 5f95076d fix(igir): bump heap to 6144MB and memory limit to 8Gi — still OOM.
  2. Batch-size shrink from 500 → 100 → 50 — still OOM, because igir's memory pressure scales with the DAT × in-flight cartesian product, not just batch size.
  3. 0fed9f37 chore(igir): suspend cronjob — OOM-crashes every run — the standing state. ROMs from sabnzbd have not been auto-curated for the last 41 days.

igir upstream has no streaming-DAT mode and no roadmap entry for one. A larger memory limit isn't a real fix — every new DAT release pushes the high-water mark back up.

Decision

Replace igir with romm-curator, a small Go program that lives in this repo at apps/media/romm-curator/. The deterministic hash-match pass (Phase 1) covers the same job as igir; a separate --enrich pass (Phase 2) hands the leftovers to the in-cluster Ollama LLM.

Why Go:

  • Single static binary, distroless final image — no Node runtime, no npm install at job time.
  • Stdlib encoding/xml + archive/zip + crypto/sha1 cover the full Phase-1 surface; no external deps in v0.1.
  • Memory footprint scales with the largest in-flight ROM, not with DAT count, because the SHA1 → game index is built once and held in a flat map.

Why in-house vs. fork:

  • Smaller surface than igir's full feature set (no MAME merged-set arithmetic, no patch chains, no NES iNES header stripping in v0.1). We can add features as we hit the need.
  • The integration points that matter to us (RomM rescan trigger, Ollama enrichment, the report layout) belong inside our infra.
  • Easier to reason about than diffing against upstream.

Alternatives considered

  • Memory bump to 16 GiB / 32 GiB. Whack-a-mole. The home-zone workers have headroom today, but every No-Intro release shifts the ceiling. We'd lose the headroom for actual workloads.
  • Streaming-DAT PR upstream to igir. Estimated effort exceeds rewriting Phase 1 in Go. Maintenance burden continues either way.
  • clrmamepro-on-cron. Closed-source, Windows-only, doesn't fit the GitOps model.
  • Just trust RomM's own scanner. RomM does a hash check against its bundled DAT subset but only for known platforms in its UI; it doesn't do 1G1R picks, region preferences, dedup-across-archives, or any of the curation duties.

Consequences

  • The new chart at apps/media/romm-curator/ ships with suspend: true so it doesn't try to run before the GitHub Actions workflow at .github/workflows/romm-curator-image.yml has built and pushed the image to GHCR.
  • apps/media/igir/ and argocd-apps/media/igir.yaml are removed in the same commit. The chart history stays in git for posterity.
  • Phase 2 LLM enrichment is the first cluster workflow that depends on Ollama for non-chat purposes. If Ollama goes degraded, Phase 1 still runs unaffected because --enrich defaults to off.
  • The RomM API key gating the rescan-after-run hook is sealed at sealed-secrets/sealed/romm-curator-creds.sealed.yaml; the chart references it via secretKeyRef with optional: true so a missing key disables the rescan call rather than failing the job.
  • Memory ceiling drops from 8 GiB (igir) to 1 GiB (romm-curator). Schedule, NFS mount points, and the sabnzbd → curator → RomM flow remain identical.

References

  • apps/media/romm-curator/README.md — rollout instructions and configuration knobs.
  • apps/media/romm-curator/cmd/romm-curator/main.go — entry point.
  • igir's OOM history: commits 0fed9f37, 5f95076d, 8ecfc1f6, d69d292c.
  • Ollama in-cluster: argocd-apps/ai/ollama.yaml.
  • Open WebUI RAG (sibling AI workload): apps/ai/cluster-rag-indexer/.