Preservation Harvest Got Less Fragile
The preservation rating run now survives search outages, strips invalid NUL bytes before storage, and has a real select-all control for bulk runs.
The preservation ratings stopped dying on boring junk
The select control finally behaves like a control. The old “Select” header was text, and clicking it sorted the table by name. Now it is a real checkbox: filter-aware, indeterminate when needed, and able to select all 32 visible rows in one click.
The worse breakage sat underneath. When external search ran out of credits, harvesting died immediately. Even the company’s own website, already first in line, never got fetched. Now a failed search query is marked as a transient outage, and everything still collectable keeps going. If the system still finds zero usable sources, it retries up to 5 times instead of burying the company after the first failure.
The second nasty bit was invisible: 13 companies failed every time on a NUL byte inside fetched text. The database correctly refused to store that garbage. Now the byte is stripped at the common point where full page text enters the system. Evidence handling has an extra guard too. Heroes, companies, grants, and citations get the same protection.
In the background, hero search can now admit 150 candidates per run instead of 100, and duplicate rows were fixed for Peter Diamandis, Laura Deming, and Jim Mellon. The bug was dull: canonical-name matching broke when the X handle existed on only one side. That is how “small” database mistakes become extra rows you then have to clean up.
📊 32 rows selected by one checkbox · up to 5 harvest retries · 13 NUL-byte failures closed · hero search 100→150 · 3 hero duplicates fixed
Read the journal
I chose the preservation rating thread because it carried the day’s clearest product story: bulk rating was possible in the interface, but two dull infrastructure edges made it fail in practice. The hero work is included briefly because it also moved today, but it was not the main arc.