The link attribute in a feed might not be unique, relying only on that to identify entries is not enough

[StayPirate]

I found that the ReadME Github podcast set the same link for most of the items. Since rss2email tracks read items via their link, this cause any new item to be consider already processed, so ignored.

[StayPirate]

Hi @skx today I find out another feed which behaves the same way. As you can see all the entries uses the same link

<item>
<title>ZDI-CAN-22550: Lexmark</title>
<guid isPermaLink="false">ZDI-CAN-22550</guid>
<link>
http://www.zerodayinitiative.com/advisories/upcoming/     <-----------------------------------
</link>
<description>
A CVSS score 6.3 <a href="http://nvd.nist.gov/cvss.cfm?calculator&version=3.0&vector=(AV:A/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:L)">(AV:A/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:L)</a> severity vulnerability discovered by 'Foundry Zero' was reported to the affected vendor on: 2023-12-01, 4 days ago. The vendor is given until 2024-03-30 to publish a fix or workaround. Once the vendor has created and tested a patch we will coordinate the release of a public advisory.
</description>
<pubDate>Fri, 01 Dec 2023 00:00:00 -0600</pubDate>
</item>
<item>
<title>ZDI-CAN-22454: Hewlett Packard Enterprise</title>
<guid isPermaLink="false">ZDI-CAN-22454</guid>
<link>
http://www.zerodayinitiative.com/advisories/upcoming/     <-----------------------------------
</link>
<description>
A CVSS score 7.8 <a href="http://nvd.nist.gov/cvss.cfm?calculator&version=3.0&vector=(AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H)">(AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H)</a> severity vulnerability discovered by 'Anonymous' was reported to the affected vendor on: 2023-12-01, 4 days ago. The vendor is given until 2024-03-30 to publish a fix or workaround. Once the vendor has created and tested a patch we will coordinate the release of a public advisory.
</description>
<pubDate>Fri, 01 Dec 2023 00:00:00 -0600</pubDate>
</item>
<item>

because of that rss2email consider any entry as already seen and it's not possible to use the feed with it.

Since it's not the first feed I see providing the same link on multiple entry, maybe a more unique way to mark entries in rss2email should be considered. What do you think?

[skx]

I guess that's something we could work around:

• If there are duplicates, then we can use the guid value.
  • Perhaps something link {{.link}}#{{.guid}} ?

I think that really the feed is a bit horrid and broken, but it should be something we can cope with.

[StayPirate]

Would such a change triggers all entries once again? I guess an initial run with -send=false should mitigate that problem.

If you proposal will be implemented, it could impact the way how the link is shown in the email client once the email is received? Or that would still be a clean X-RSS-Link: {{.Link}}?

Another solution I can think of is to calculate a CRC or a quick hash of the entry and track that one on the rss2email's seen-database. But that could be slower (because of the hash calculation) and trigger new emails in case old items change over time, like an updated description will trigger the email notification once again.

[skx]

Would such a change triggers all entries once again?

Yes

I guess an initial run with -send=false should mitigate that problem.

Yes

If you proposal will be implemented, it could impact the way how the link is shown in the email client once the email is received? Or that would still be a clean X-RSS-Link: {{.Link}}?

Yes, the link would be literally updated.

Another solution I can think of is to calculate a CRC or a quick hash of the entry and track that one on the rss2email's seen-database. But that could be slower (because of the hash calculation) and trigger new emails in case old items change over time, like an updated description will trigger the email notification once again.

I think I'd probably not love that approach, but I guess it is an option.

Does #120 work for you, or is that just a doomed approach?