filex

Appearance

Thumbnails ​

filex renders preview thumbnails server‑side for images, video, audio, PDFs, office documents and SVGs, and a coloured placeholder card for everything else. Thumbnails are on by default — the grid, the list and the gallery all show a real preview where one exists and fall back to a per‑type icon where it doesn't. (Thumbnails were a grid‑only feature for a long time; the list drew a type glyph for every row, including a photograph.)

The image and placeholder generators are pure Go and always work. The richer kinds (video, audio, PDF, office, SVG) each shell out to an external tool that filex auto‑detects on PATH at startup — if the tool is missing, that kind degrades gracefully instead of erroring.

How it works ​

The pipeline (backend/internal/thumb/) is a dispatcher: it inspects each file node's MIME type — falling back to the file extension when the MIME is empty, which is the common case for files discovered by a storage sync — and routes it to exactly one generator.

Every generator writes a JPEG to the cache directory as <cache_dir>/<nodeID>.jpg (regardless of source kind, the cache file is always <id>.jpg, roughly 320 px on the long edge) and updates a row in the thumbnails table with a state:

StateMeaning
pendingDispatched, not finished yet (or left over from a crash).
readyA JPEG is cached and servable.
skippedNo generator applied (e.g. SVG with no rsvg-convert). Not an error.
failedA generator ran but errored (broken file, tool crash). Logged at WARN with the error stored on the row.

Generation is triggered two ways:

  1. After a write — the moment an upload, a public file‑drop or a document created from + New commits, filex dispatches the pipeline in a detached background goroutine with its own timeout (90 s on the browser upload path, 2 min on the staged, AI/REST and protocol paths — an office→PDF conversion of a large deck is the reason the longer ones exist). The HTTP request returns immediately; a client disconnect can't abort an in‑flight conversion. Errors are swallowed (the pipeline logs its own).
  2. Backfill — a one‑shot pass over files that already exist in the cache (see Backfill).

Cached JPEGs are released two ways, both described in Reclaiming the cache.

Generators & required tools ​

KindSource typesGeneratorExternal binary (auto‑detected on PATH)
Imageimage/* — jpg, png, gif, bmp, tiff, webpBuilt‑in Go (stdlib + x/image)none
Videovideo/* — mp4, webm, mov, mkv, avi, …ffmpeg — the first frame that is not black, searched over the opening 10 s, scaled to 320 wideffmpeg
Audioaudio/* — mp3, wav, ogg, flac, m4a, aac, opusffmpeg — a 320×120 waveform image (showwavespic)ffmpeg
PDFapplication/pdfGhostscript renders page 1 at 96 dpi (falls back to poppler), then the page is scaled down to thumbnail sizegs or pdftoppm
Officedoc, docx, xls, xlsx, ppt, pptx, odt, ods, odpLibreOffice headless → PDF → page 1 through the same renderer the PDF path useslibreoffice (or soffice) and one of gs / pdftoppm
SVGimage/svg+xmllibrsvg rasterises to PNG → re‑encoded to JPEGrsvg-convert
Placeholdereverything else — archives, 3D models, code, markdown, rtf, raw docs, …Built‑in Go — a tinted card with the extension centred (colour hashed from the extension)none

Notes:

  • Images decode with the Go standard library plus golang.org/x/image (BMP / TIFF / WebP), capped at ~50 MB of decoded input, and are downscaled to fit 320×320 (aspect preserved; larger sources only) and encoded at JPEG quality 80. Formats Go can't decode — e.g. HEIC / AVIF — will state=failed.
  • SVG is checked before the generic image/* branch, because Go's decoder can't parse SVG. If rsvg-convert isn't present the SVG is cleanly skipped, never failed.
  • Office goes through two tools: LibreOffice to make a PDF, then Ghostscript/poppler to rasterise page 1. It also wants a JRE and fonts present for reliable conversion (the stock full image ships both).
  • Video: "first frame" means the first one with something in it. A great many real clips open on black — a fade‑in, a slate, a camera's leader — so filex asks ffmpeg for the first frame whose average luma clears 24 (video black is 16, not 0) within the opening 10 seconds, and falls back to the literal first frame when the whole opening is dark, because then black really is what the video looks like. This replaced a single -ss 1 seek that failed two measured ways: a clip shorter than a second decoded no frame at all while ffmpeg exited zero (the row said ready and the card 404ed), and a 1.6 s fade‑in produced a 581‑byte pure‑black JPEG. Every ffmpeg and Ghostscript run is now followed by a check that a file actually came out — an exit code does not tell you whether a frame did.
  • The PDF and office paths are one renderer (renderPDFPage1). office.go used to carry its own transcription of the gs/pdftoppm block, which is how it kept bugs the PDF path had already been fixed for — the missing downscale, the zero‑exit‑but‑no‑file case, and pdftoppm's zero‑padded output name (page 1 of a 120‑page file is ‑001.jpg, not ‑1.jpg, so the old rename silently missed on any PDF with ten or more pages).
  • A rasterised page is downscaled like everything else. gs -r96 renders A4 at 816×1056, and that used to be what the cache handed out: a 358 KB JPEG drawn inside a 184×108 card, for every PDF and office document in a folder. Pages now go through the same fit‑to‑320 step the image generator uses. In the card a page is anchored to its top rather than centre‑cropped, so the letterhead and title — the only part that says which document it is — stay visible.
  • Text, code and CSV never reach the placeholder in the explorer. The backend will happily render them an extension tile, but the grid, list and gallery skip the request and draw the file's own first lines instead: a ranged read of at most 8 KiB, only once the card is on screen, only for files under 4 MB, cached per path and version. It is the same one request the generic card cost, spent on something that says what the file is.

Configuration ​

SettingDefaultWhereMeaning
FILEX_THUMBS_ENABLEDtrueenvMaster switch. Accepts 1 or true (case‑insensitive) as on; any other value is off.
FILEX_THUMB_BACKFILL_ON_BOOT(unset)envSet once (or true / 1) to run one background backfill on startup. See Backfill.
thumbs.cache_dir<data_dir>/thumbsconfig.yaml onlyDirectory the cached <id>.jpg files live in. No env override.
thumbs.formats[image, video, pdf, office]config.yaml onlyDeclares the kind list. No env override.
FILEX_THUMBS_SWEEP_INTERVAL6henv / thumbs.sweep_intervalHow often the cache is reconciled against the node catalogue. 0 disables the sweeper entirely. See Reclaiming the cache.
FILEX_THUMBS_URL_TTL24henv / thumbs.url_ttlHow long a stamped thumb_url stays valid — see Serving. Matches the endpoint's Cache-Control: private, max-age=86400. ⚠ 0 means use the default, not "never expires"; an unbounded stamp would be a permanent bearer capability for that preview. Shortening it never locks out the SPA, the desktop app or an embedded explorer — all three fetch with credentials and are authorized per request.

There is no env var or config key for the external tools — filex probes PATH at boot (ffmpeg, gs, pdftoppm, libreoffice/soffice, rsvg-convert) and enables each kind accordingly. In practice a kind renders when its MIME type matches and its tool is present; cache_dir is the thumbs.* value read at runtime.

Reclaiming the cache ​

A thumbnail outlives nothing: when its file is gone for good, so are its bytes.

At the moment of deletion. Purging a file — emptying the trash, a retention expiry, or "delete permanently" — removes its <id>.jpg and its thumbnails row there and then, so the space comes back when the user asks for it.

⚠ Trashing a file does not. A trashed file is restorable and keeps its thumbnail, so it is on screen the instant it comes back.

The sweeper. Every FILEX_THUMBS_SWEEP_INTERVAL (and once at boot) filex walks the cache directory and deletes files whose node no longer exists. This is what repairs an install that has been accumulating orphans — from a removed storage, a sync tombstone, or simply from a version of filex that never cleaned up at all — and it logs one line per pass, including the passes that delete nothing:

thumb cache sweep dir=/data/thumbs scanned=20412 removed=317 freed_bytes=6114233 kept=20095 skipped=0 interval=6h0m0s

A file is deleted only when all of the following hold, which is what makes the sweeper safe to run unattended:

  1. its name is exactly <digits>.jpg — nothing else in the directory is ever a candidate, so a file you put there yourself is left alone;
  2. the database positively reports that node id absent from nodes. A trashed node still has a row. If the query fails, the pass is abandoned and nothing is deleted — "I could not ask" is never read as "it is gone";
  3. node ids are never reused (AUTOINCREMENT on SQLite, BIGSERIAL on Postgres), so an id that is absent today cannot acquire a file tomorrow;
  4. the file has not been written within the last 10 minutes, so a thumbnail still being generated is never judged mid‑flight.

Set FILEX_THUMBS_SWEEP_INTERVAL=0 to turn it off; nothing else in filex removes a cached thumbnail on a schedule.

The Docker image & bundled tools ​

Image thumbnails and placeholder cards work on any image, including the smaller :slim image, because they need no external binary.

The default ghcr.io/brf-tech/filex:latest image bundles the tools that unlock the richer kinds:

ffmpeg          → video + audio thumbnails
ghostscript     → PDF (page 1)  ┐ office docs render via
poppler-utils   → PDF fallback  ┘ LibreOffice → PDF → these
libreoffice     → doc/docx/xls/xlsx/ppt/pptx/odt/ods/odp
openjdk17-jre   → LibreOffice's conversion pipeline
rsvg-convert    → SVG
imagemagick     → reported as `thumbs.imagemagick` in the capabilities probe
fonts (noto/liberation/dejavu)  → so office/PDF text isn't rendered as boxes

The :slim image deliberately ships none of them (they are ~470 MB together, and they are the reason two images exist). Whatever image you run, the definitive check for what is actually present is the capabilities probe (thumbs.svg, thumbs.video, …) — and the boot log, which now names every tool that is missing (see What happens if it isn't configured).

If you build your own leaner image, drop tools from the install list — the capability probe will report video=false / pdf=false / etc. and the pipeline routes around the missing generators automatically.

Serving ​

GET /api/files/thumb/{id}

  • Returns 404 unless the node's thumbnail state is ready and the cached JPEG exists on disk.

  • On success: Content-Type: image/jpeg and Cache-Control: private, max-age=86400 (cache for 1 day).

  • Authorized, by one of two proofs. A bad id returns 400; no proof at all returns 401.

    1. A live stamp on the URL — ?exp=<unix seconds>&sig=<hex hmac>, an HMAC‑SHA256 over "<id>.<exp>" under the thumb_signing_key setting (generated on first use). This is what a bare <img src> can carry: it sends no Authorization header, and the session cookie is SameSite=Lax so it is not sent by an <img> inside a third‑party embed either.
    2. An authenticated caller — session cookie or bearer/API token — who passes the node's tenancy scope, the token's root: confinement and an ACL check at viewer level. A node the caller cannot reach answers 404 (the same answer as a node that does not exist, so the endpoint is not an enumeration oracle); a node they can see but not read answers 403.

  • File listings include a thumb_url per node, already stamped — the listing is the only place that knows the caller was allowed to see that node, so it carries the decision forward into the URL.

⚠ Before the release this note ships in, the sig parameter was optional and the signing key was never written, so GET /api/files/thumb/{id} served a rendered preview of any file on the instance to any anonymous caller who could guess a node id — on single‑tenant installs too. If you run an older build, put it behind authentication at the reverse proxy or upgrade.

⚠ The stamp is a capability, not an identity: whoever holds the URL can fetch that one node's preview until exp. That is the same trade a share link makes, and it is what makes a header‑less <img> possible at all. FILEX_THUMBS_URL_TTL bounds it.

⚠ The public folder‑share page does not use this endpoint. It serves the same cached artefact through /s/{token}/f/<path>?thumb=1, scoped to the share token, so an anonymous share viewer needs no stamp and no session.

Capabilities (used by the UI and handy for debugging) are exposed at GET /api/files/capabilities (legacy alias GET /api/capabilities) under thumbs:

bash
curl https://files.example.com/api/files/capabilities | jq .thumbs
json
{ "image": true, "imagemagick": true, "video": true, "audio": true,
  "pdf": true, "office": true, "svg": true }

(Every one of them is true on the stock full image and false except image on :slim.)

(The probe result is cached for 1 hour.)

Backfill — catching up existing files ​

New uploads get a thumbnail automatically. Files that entered the cache another way — a storage sync, or an install that previously ran without the tools — do not, so their rows stay empty. The thumb backfill command walks every file node and (re)dispatches the pipeline:

bash
filex thumb backfill                     # every enabled storage
filex thumb backfill --storage local     # one storage, by name
filex thumb backfill --storage 2         # one storage, by id
filex thumb backfill --limit 100         # stop after 100 files (across all storages)
filex thumb backfill --retry-failed      # also re-run rows in state=failed
filex thumb backfill --retry-skipped     # also re-run rows in state=skipped
filex thumb backfill --concurrency 8     # worker pool size (default 4)
filex thumb backfill --progress-every 50 # progress line every N files (default 25)

Which files are (re)processed:

Existing stateRe‑run?
(no row) / pendingAlways.
readyNever (idempotent).
skippedOnly with --retry-skipped.
failedOnly with --retry-failed.

The walk skips trashed and soft‑deleted nodes. It ends with a summary line — {processed: N, ok: M, failed: K, skipped: S} — and exits non‑zero only on infrastructure errors (DB unreachable, unknown --storage, …); per‑file failures are counted into failed but don't abort the run.

⚠ Search index lock. A running filex serve holds an exclusive lock on the Bleve (boltdb) search index. Backfill never touches search, so it disables the index for its run (sets FILEX_SEARCH_ENABLED=false unless you've already set it) — otherwise it would block indefinitely acquiring that lock. Only override FILEX_SEARCH_ENABLED=true when running backfill on a stopped node.

Boot-time backfill ​

For containers where you want each restart to make sure the grid is painted:

FILEX_THUMB_BACKFILL_ON_BOOT=once

(values once, true, 1 are equivalent; anything else leaves it off). When set, serve launches one background backfill a couple of seconds after the HTTP listener is up — so the boot path stays fast — and logs progress at INFO via slog (thumb backfill (boot): starting one-shot backfill). It's off by default; most operators prefer to trigger backfills explicitly.

What happens if it isn't configured / a tool is missing ​

  • Thumbnails are on by default. With zero external tools you still get real image previews plus placeholder cards for everything else.
  • A missing tool says so at boot, by name. A grid of coloured rectangles looks like a design choice rather than a missing package, so nobody goes looking for the package — and two of the three ways filex ships (:slim, the bare binary) arrive with none of these tools. With everything present the line is INFO (thumbs: every preview kind available …); otherwise it is a WARN naming each unavailable kind and what would install it:
    WARN thumbs: some previews will fall back to a plain type tile; the tool that
     draws them is not installed  unavailable=audio,office,svg,video
     install="audio needs ffmpeg; office needs libreoffice; svg needs rsvg-convert; video needs ffmpeg"
  • Missing tool for video / audio / PDF / office → that kind can't be enabled, so the dispatcher routes the file to the generic placeholder card. The state is ready, not failed — the grid shows a legible tinted card with the extension, just not a real preview.
  • SVG with no rsvg-convert → state skipped (reason: rsvg-convert not in PATH). No placeholder is drawn; the UI shows its own SVG icon.
  • A generator that runs but errors (tool present, but the file is broken / truncated / unsupported) → state failed, a WARN is logged, and the error text is stored on the row.
  • Unsupported / other kinds (archives, 3D models, rtf, raw docs, …) always get the placeholder card (ready). Text, code and CSV files get one too, but the explorer does not fetch it — it draws their first lines instead (see Generators).

Failure modes & troubleshooting ​

The grid shows icons, not previews ​

The thumbnail isn't ready. Inspect the thumbnails table:

bash
sqlite3 <data_dir>/instance.sqlite \
  "SELECT node_id, state, error FROM thumbnails ORDER BY node_id DESC LIMIT 20;"

ready rows serve a JPEG; failed rows carry the generator error in error; skipped/absent rows fall back to the per‑type icon.

Existing files never got thumbnails after I added the tools ​

Uploads generate automatically, but files already in the cache don't. Run filex thumb backfill (or set FILEX_THUMB_BACKFILL_ON_BOOT=once). If a whole storage is empty, run a sync first — backfill only walks nodes already in the cache.

Office documents land in state=failed ​

LibreOffice converted but the second stage failed, or LibreOffice itself did. Usual causes: no PDF renderer (gs/pdftoppm missing → LibreOffice succeeds but there's nothing to rasterise the PDF), a missing JRE or fonts, or a corrupt/truncated source doc. The stock full image already ships the JRE and fonts; check the error column for the LibreOffice/Ghostscript output.

SVGs never render ​

rsvg-convert isn't on PATH — the capabilities probe shows thumbs.svg:false and rows are skipped. The stock full image ships librsvg, so this is a :slim, a bare‑binary or a custom image; install it (apk add rsvg-convert) and re‑run with --retry-skipped.

PDF or video previews are blank / missing ​

If the tool is entirely absent the file becomes a placeholder (ready), not a failure. If the tool is present but the row is failed, read the stored error — a broken PDF, an unreadable codec, or a permissions issue on the temp dir. A run that exits zero without producing an image is now caught rather than stored as ready: the error names it (ffmpeg exited 0 but wrote no frame, pdftoppm exited 0 but wrote no page) and carries both renderers' own output, because gs and pdftoppm fail for different reasons and only one of them usually prints why.

HEIC / AVIF images fail ​

Go's decoder only handles JPEG, PNG, GIF, BMP, TIFF and WebP. HEIC/AVIF sources end up failed. Convert them, or add an external converter upstream.

Backfill seems to hang ​

It's almost certainly the search‑index lock — see the callout above. Backfill disables search for its run by design; don't force FILEX_SEARCH_ENABLED=true while filex serve is live.

A regenerated file shows the old thumbnail ​

The cache lives at <data_dir>/thumbs/<id>.jpg and is safe to delete. Remove the stale file (or the whole thumbs/ dir) and re‑run filex thumb backfill --retry-failed — the cache is regenerated lazily.

⚠ This also covers a case nobody does by hand: a file replaced on the backing storage, which the storage sync now notices on every driver. The sync updates the row, the search index and the antivirus verdict — and does not touch the cached JPEG, so the grid keeps showing the old picture until something removes the file. Same fix: delete <data_dir>/thumbs/<id>.jpg.

See also ​

Released under the MIT License.

© BRF Tech · filex.sh