Keep discovery and verification visible
A feed reader is an efficient place to discover a new document. Turning that discovery into an article requires a richer record. An editor needs to know where the item came from, what the linked document actually says and when that document was examined. A headline and a timestamp leave much of that work unrecorded.
The RSS 2.0 specification defines an optional item GUID that an aggregator may use to identify whether an item is new. A GUID can be an arbitrary string; its permalink interpretation depends on the relevant attribute. Treating every GUID as a fetchable article address would therefore merge two distinct fields before any editorial work begins.
Atom's format specification defines a permanent identifier and also provides separate publication, update and source metadata. These fields are useful inputs to a ledger. They do not certify that an author's assertion is correct, that a timestamp reflects independent verification, or that an updated entry says the same thing as its earlier version.
Preserve enough detail to explain a revision
Our suggested record keeps the feed address, feed item identifier, original linked URL, final resolved URL and retrieval time separately. Add the source document's relevant version when it has one. Preserve a bounded evidence excerpt or an appropriate internal capture reference, together with a digest of the bytes used for the check. Avoid retaining unnecessary personal or restricted material.
HTTP metadata can help reduce repeat work, but it has its own meaning. RFC 9110 describes entity tags as opaque validators selected by the server. An ETag is not automatically a cryptographic hash of the editorial excerpt. Keep the HTTP validator and your own evidence digest as separate fields rather than allowing one to impersonate the other.
Deduplicate without erasing the reporting trail
Several feeds may announce the same release, while one publisher may revise an item in place. A practical ledger can group these discoveries under one candidate topic without deleting their individual origins. That lets an editor see whether apparent corroboration consists of independent reporting or several links to the same announcement.
URL normalization should also be conservative. Removing a tracking parameter may be reasonable for a known publisher. Removing every query string can combine different documents on another site. Keep the original address alongside any normalized key and record the rule used. When the rule changes, the original discovery should remain understandable.
For a revised source, compare the claim-supporting material before deciding whether an existing draft needs attention. A changed navigation menu and a corrected product limitation should not create the same editorial task. A digest can reveal that bytes changed; a person or bounded review process still needs to determine whether the difference matters to the article.
Publish an account the reader can follow
The public article can link to the primary source, state which version informed the analysis and make corrections visible. The internal ledger can retain the more detailed acquisition and claim history. Readers usually need a clear trail to the evidence rather than every fetch log.
This arrangement preserves RSS's useful simplicity. Existing feed software handles discovery and update checks, while the editorial ledger records the additional judgment required for publication. The result is a process in which an editor can explain both how a topic arrived and why the eventual article says what it does.
The experiment we actually ran
On November 6, 2026, we tested feedparser 6.0.14 in an isolated Python 3.13.7 environment on Darwin/arm64. This established feed parser is released under BSD-2-Clause; its installed dependency was feedparser-sgmllib 2.1.0. We used an existing parser rather than building an RSS ingestion framework.
The recorded capture of our public RSS feed returned HTTP 200 and 9,144 bytes. Parsing those saved bytes identified RSS 2.0, returned 18 entries and reported bozo=False. The captured file and its SHA-256 are included in the downloads below.
All 18 identifiers happened to equal their corresponding article links. However, their GUID permalink flags were false. A string resembling a URL therefore did not establish that the identifier was intended as a fetchable address. We retained the parsed identifier and link separately. We downloaded the feed, without retrieving the linked articles or checking their claims.
Repeat the input, keep both observations
We parsed the same saved XML twice, using two input labels. These were repeated local inputs, not independent live observations. The two parser calls produced 36 entry observations. feedparser did not remove the repeated records for us.
The downloadable script adds our own small editorial_ledger example. It groups by feed address plus item identifier, falls back to the exact link when an identifier is absent, and retains every observation. An entry without either receives a separate observation identity. This is illustrative editorial logic, not a built-in feedparser deduplication feature.
For the repeated public input, the example produced 18 identity groups, each containing two observations and one parsed revision. Its revision fingerprint covers selected parsed fields, including title, link, summary and dates. That fingerprint differs from the raw-feed digest and from an article evidence digest. A matching fingerprint only tells us those selected fields stayed equal; it does not establish that the linked article was unchanged. Linked-document URLs after redirects and document digests remain empty because that retrieval was not run.
Synthetic edges that change the decision
The public snapshot was orderly, so we also executed ten explicitly synthetic inputs using reserved .example addresses. None of these fictional publishers or URLs was fetched. The cases expose decisions a quiet, valid feed cannot test:
- An opaque GUID: the parser preserved
urn:example:record:42separately from its HTTPS article link. The ledger can identify an entry without treating its ID as a destination. - A revision with the same ID: a fictional request cap changed from 10 to 8. Our example retained one identity with two parsed revisions. It did not decide whether the change justified a correction or publication.
- A common link and different queries: two fictional feeds pointing to one document remained two identities, with their shared link visible. Separately, URLs with
edition=1andedition=2remained distinct. No blanket query-string removal was performed. - Unknown dates and identities: an invalid publication date retained its original text and had no parsed date. Two entries with the same title but no ID or link remained separate observations. We did not invent a timestamp or merge them by headline.
- Broken input: deliberately truncated XML set the bozo flag while still yielding one recoverable entry. Our example held that batch for review. Ordinary HTML produced no recognized feed version and no entries, and was also held. A recovered record or an unset bozo flag alone was insufficient for acceptance.
Replay it and keep the boundary visible
The recorded experiment passed 22 checks. A second offline run reproduced the complete outcome object. A further replay with socket connection and DNS operations blocked also reproduced it. That demonstrates this replay's independence from live network responses; it does not establish production ingestion reliability.
The saved run measured 25 cached-byte parses with a median of 1.68 milliseconds on this machine. This excludes fetching, article retrieval and editorial judgment. It is a small local measurement, not a VPS load test, cross-library ranking or promise about your hardware.
To reproduce the experiment, download the replay instructions, Python script as text, pinned requirements, public feed snapshot and synthetic fixtures. The script reads those saved inputs; installing the pinned packages is a separate setup step.
For the evidence, inspect the recorded results and capture receipt. They retain the checked fields, assertions, timing and input fingerprints.
Conditional requests and HTTP 304 behavior, linked-document verification, semantic change review, unattended scheduling and production publication were not tested. No model was called. This experiment provides inspectable discovery records and demonstrates conservative grouping; the additional evidence and judgment needed to publish remain separate work.