The development of RTD framework within Prebid Server

[TechDevCrew]

Dear community!

Could you please give your attention to the question.

There is real-time data (RTD) module within Prebid.js with a lot of sub-modules, but it seems that such functionality is absent for the Prebid Server.

I want to develop a RTD framework within Prebid Server.
Later - RTD module for this framework, that will allow for the injection of data from 3rd party services into bid requests before they reach bid adapters in Prebid Server.
Alike it was implemented within Prebid.js.

I’d like to understand are there any concerns about it? Also are there any prerequisites?
Please provide your considerations and any help with starting points of the development this feature.

Thanks in advance.

[bretg]

Glad there's interest here @TechDevCrew . Are you a prebid.org member? We meet every two weeks to discuss topic like this.

Anyhow, my thoughts:

1. In some ways, RTD modules can already be built in PBS-Java using the new modularity infrastructure! The first one is being built right now: Price Floors.
2. I'd propose that in order for PBS-Go to support RTD modules, the first step is to port the modularity infrastructure PRD.
3. Since making RTD modules may be difficult, we could define what PBS services could be useful to RTD module authors. Here's a cut:
   1. We could define a new dynamic configuration path that's not account-related. i.e. create a "feature" table in the DB so parameters can be updated with a deploy and bounce. RTD modules can then poll their config for updates or we could implement a service that invokes a function when something's changed.
   2. PBS could provide a standard way for modules to invoke their data endpoint that makes use of metrics and circuit breakers. (maybe this is already available?)
   3. PBS could to provide a standard way for RTD modules to atomically update their data results so that module hooks can make use of the data without worrying about read conflicts.
4. But I think a RTD module could be built right now in PBS-Java, or if it couldn't would like to know what requirements are missing.

We're working on the tech spec for Price Floors and came up with this general diagram:

The general idea is that periodic data fetching is quite different computationally from the main auction activity. We need to minimize the interaction between those parts of the server.

[TechDevCrew]

Dear @bretg!
Thank you so much for so comprehensive answer!
Could you or colleagues pay attention to my draft of pull request.
Does it the right way of developing?
Please share your suggestions, considerations about development of this feature.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[bretg]

Copying the comment from the PR

The Prebid Server Committee discussed RTD functionality and we agree with @bretg that this would be best suited for after the #1734. The hooks described in that specification are very likely sufficient for RTD if we add helper methods or framework for buffering and sending data to RTD servers.

Closing this issue.