Activity feed: JSON API

[rouk1]

As a frontend developer I want to request the project's activity using an HTTP API.

Disclaimer: even if this a perfect use case for web sockets, we will stick to a REST endpoint.

Needs

We need to serialize a list of item version for the frontend to display them from most recent to oldest one.

Note

This means that: if a user asks for the last 5 minutes of activity, and one item has changed 2 times during this time frame (same identifier, two different values), they will receive 2 versions of this item.

Proposed API

GET /api/project/activity?after={creation_datetime} (fetch the project activity)

Parameters

name	type	data type	description
after	optional	datetime as UTC string (iso 8601)	A search filter (every item that have been added after the given datetime). Adding this parameter will allow frontend to only request new data after the initial call.

Response

http code	content-type	response	description
200	application/json;charset=UTF-8	[{...}, {...}, ...]	A list of item ordered by descending create_at datetime.

Data layout

[
  {
    "name": "md_item",
    "media_type": "text/markdown",
    "value": "# hello",
    "created_at": "2020-07-10 15:00:00.000"
  },
  {
    "name": "df_item",
    "media_type": "application/vnd.dataframe+json",
    "value": {},
    "created_at": "2020-07-10 14:00:00.000"
  },
  //...
]

[rouk1]

@augustebaum & @thomass-dev may I have your review on this ?

[tuscland]

I have a few questions:

• The spec includes support for warnings and potentially other types of objects. The proposed API does not seem to cover this.
• Can we still leave without identifiers for items or anything that is stored? It feels like we are pushing the model to its limits for nothing.

[tuscland]

Note: warnings are removed from this first iteration.

[rouk1]

• Can we still leave without identifiers for items or anything that is stored? It feels like we are pushing the model to its limits for nothing.

Hmmm your right I forgot that. Editing...

[rouk1]

Note: warnings are removed from this first iteration.

It could be quite easy to add warnings.
A warning could have a media_type and be persisted in a dedicated repository (like we did for views).

[
  {
    "name": "High class imbalance",
    "media_type": "application/vnd.warning+text",
    "value": "It seems that you have a classification problem with at least one class with...",
    "updated_at": "2020-07-10 15:00:00.000",
    "created_at": "2020-07-10 15:00:00.000"
  }
  //...
]

[augustebaum]

Does after refer to created_at or updated_at? Should there be created_after and updated_after parameters instead?

[augustebaum]

Disclaimer: even if this a perfect use case for web sockets, we will stick to a REST endpoint.

Can you give more details? Why is it a perfect use-case, and if so, why would we not do that?

[rouk1]

Does after refer to created_at or updated_at? Should there be created_after and updated_after parameters instead?

It's a good question. Those naming comes directly from what we have in the project API. I prefer to have consistent naming for the same thing. But you're right to have consistent data and as each item will have only one creation_date we can change the data layout to:

{
    "name": "High class imbalance",
    "media_type": "application/vnd.warning+text",
    "value": "It seems that you have a classification problem with at least one class with...",
    "created_at": "2020-07-10 15:00:00.000"
}

and avoid confusion.

Disclaimer: even if this a perfect use case for web sockets, we will stick to a REST endpoint.

Can you give more details? Why is it a perfect use-case, and if so, why would we not do that?

It's an activity feed. The term feed could let you think that it's real time. Hence a real time communication protocol sounds appropriate. But to implement this without some message broker we would need to watch the db changes. As far as I know, because we use diskcache, we cant have easily access to the sqlite driver to register some hooks. As a first approach polling with incremental search will have good performances and abstracts us from complex communication between process.

[augustebaum]

each item will have only [a] creation_date [not a update_date]

Works for me

[rouk1]

edit: added @augustebaum feedbacks

[thomass-dev]

Just a naive question: what about mixing history and activity?

If you ask for the last 5 minutes activity, and one item has changed 2 times during this period (same identifier, two different values). Would you receive the two versions of the items? Or the latest one?

Otherwise, perfect proposition, thanks @rouk1 .

[rouk1]

Just a naive question: what about mixing history and activity?

If you ask for the last 5 minutes activity, and one item has changed 2 times during this period (same identifier, two different values). Would you receive the two versions of the items? Or the latest one?

You should receive all events that happens during that time frame. Hence 2 versions of your item.

[thomass-dev]

Can you update your description that way and make a quick reference?

[rouk1]

Can you update your description that way and make a quick reference?

Done, is it ok for you ?

[tuscland]

LGTM