[FEATURE] Publish RSS Feed Action

[chrigi]

Is your feature request related to a problem? Please describe.
I tried to create a workflow I have done quite a few times on Yahoo Pipes and on my Huginn instance:
Importing events from an RSS feed, filtering them and then publishing the filtered items as a new RSS feed.

This is currently not possible in a straightforward way on PD as you can't publish an RSS feed and at the same time define an RSS trigger for the same workflow and you can't share data between workflows directly.

Describe the solution you'd like
I would like a "Publish RSS Feed" action with params to add feed metadata and data input from previous steps. It should then aggregate this data up to a configurable size (e.g. 50 items) and when deployed generate a URL to be callable where this feed is being served from. When more than the defined amount of items accumulate it should drop the oldest items.

This action should be able to be part of any kind of workflow, regardless of what trigger is used.

There are various details to be considered but not that important for this feature request like: Concurrency/Race-Condition, RSS, ATOM or JSON support, input data format, Content-Type header, etc.

Do you have a workaround?
There are 2 ways I can see how I can make it work with the current PD systems:

1. Use 2 workflows and an external DB. One workflow would use an RSS trigger to read the items & filter them, then write them to a DB. The second workflow would have a HTTP API Trigger and implement something similar to @dylburger example here: https://pipedream.com/@dylburger/generate-an-rss-feed-from-http-post-requests-retrieve-via-get-request-p_n1CrQG/edit (just removing the adding of items with POST)
2. I could use the mentioned example as the main workflow and replace the logic in the POST step with loading & filtering the desired feed. Then I would create a second time triggered workflow to call the main workflow with a POST request to trigger creation of the checkpoint data.

Number 1 is nice but requires a 3rd party DB service and plumbing around that. Also with the shutdown of the SQL service I'd assume PD is not interested in adding a new DB service to their platform to make this easier.
Number 2 is incredibly ugly and when trying it I constantly ran into timeouts for larger feeds.

Comparable features in other tools?
Huginn: https://blog.andrewcantino.com/blog/2014/04/13/adding-rss-feeds-to-any-site-with-huginn/
Yahoo Pipes: https://arstechnica.com/information-technology/2009/03/yahoo-pipes-getting-started-with-custom-rss-feeds/

Additional context
PD Community discussion: https://pipedream.com/community/t/publish-rss-feed-action/1856/2

I know many people think RSS is dying but I still consider it very valuable and many tools still work with RSS as their main ingest pipeline.

This could in theory be extended to a more generic solution, allowing actions to generate callable URLs when deployed and serving some internal state there. This would require considerations regarding race conditions and persistent storage of step-level data.

[dylburger]

@chrigi Recent improvements make this use case simpler. You can do approach #1 with Pipedream data stores, which are accessible across workflows. Specifically, you would have two workflows:

1. Write new items to a data store
2. Respond to GET requests, reading items from the data store and returning a well-formed RSS feed

Given the current Pipedream primitives, this is the recommended way to solve this use case. That is, actions are akin to functions. They receive input and produce output. But they aren't directly attached to triggers (HTTP, Timer, etc). For that you need to create a workflow with an associated trigger.

Let me know if that helps!

[chrigi]

@dylburger Thanks a lot, yes, I think this fulfills my request.

[eugenesvk]

@chrigi have you managed to recreate this workflow with Pipedream? Any chance you could share an example? Thanks