[GraphQL] Mercure Subscription Support #3321

alanpoulain · 2019-12-20T17:04:14Z

Q	A
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	no
Tickets	fixes #3124
License	MIT
Doc PR	api-platform/docs#1034

GraphQL Subscriptions in API Platform will use Mercure as the underlying protocol (for the first version) with this PR.

Only the update subscription is available. Maybe the delete subscription would be useful too but it could be done later.

A subscription can be done like this:

subscription {
  updateDummyMercureSubscribe(input: {id: "/dummy_mercures/1", clientSubscriptionId: "myId"}) {
    dummyMercure {
      id
      name
      relatedDummy {
        name
      }
    }
    mercureUrl
    clientSubscriptionId
  }
}

In the JSON response (in mercureUrl field), you will receive an URL like this:
https://demo.mercure.rocks/hub?topic=http://example.com/subscriptions/fc806a60cca4e93544d555912ce8b549

This URL is used by the Mercure protocol to subscribe to updates by using Server-sent events:

const eventSource = new EventSource(mercureUrl);

// The callback will be called every time an update is published
eventSource.onmessage = e => console.log(e); // do something with the payload

To enable subscriptions for a resource, follow the related Mercure documentation. It mainly consists of adding this annotation:

use ApiPlatform\Core\Annotation\ApiResource;

/**
 * @ApiResource(mercure=true)
 */
class Book
{
    // ...
}

You will only receive the update if the payload you have given when subscribing has changed.
To do so, a Symfony cache is used to store the subscriptions and their payloads:
api_platform.graphql.cache.subscription.
The best way to store the subscriptions is to use an adapter like Redis.

An Apollo transport like this one: https://github.com/CodeCommission/subscriptions-transport-sse could be written later in order to make it work seamlessly with Apollo.

alanpoulain · 2019-12-21T14:02:31Z

@lukasluecke @mahmoodbazdar @CvekCoding if you could have a look at this PR?

lukasluecke · 2019-12-21T15:39:58Z

@alanpoulain Thanks for your effort! I will check it out in the next days 🙂

CvekCoding · 2019-12-21T18:09:27Z

@alanpoulain wow, great. I'll check this on NY vacation (at least I'll try:)).

Great thanks for this update - sounds great. As far as I know it's a first graphql subscription implementation in php world.

lukasluecke · 2020-01-06T11:23:04Z

@alanpoulain Looks good so far - I'll see if I can find the time this week to try it out (i.e. build a working client-side integration utilizing this).

lukasluecke · 2020-01-06T11:25:24Z

src/GraphQl/Subscription/SubscriptionManager.php

+        foreach ($subscriptions as [$subscriptionId, $subscriptionFields, $subscriptionResult]) {
+            $resolverContext = ['fields' => $subscriptionFields, 'is_collection' => false, 'is_mutation' => false, 'is_subscription' => true];
+
+            $data = ($this->serializeStage)($object, $resourceClass, 'update', $resolverContext);


@alanpoulain This means that the subscription data will get serialized using the update context, right? Should there be a way to override it for the subscription payload specifically?

I think it would be a bad idea to have a subscription context. Subscriptions are here to send the updated data like you would receive it if you made a mutation and read the received data.

You might be right 😉 what about the case where there is no update mutation on the resource? (It can still be changed from other parts of the API, right?)

Would you see the item_query context instead then?

@alanpoulain Sorry for the late reply - I think trying update first and falling back to item_query would be good, wdyt?

Actually, if you don't have an update operation, the subscription fields are not generated, so it makes sense to me to keep it like this.

lukasluecke · 2020-01-06T11:26:56Z

src/Bridge/Doctrine/EventListener/PublishMercureUpdatesListener.php

+        foreach ($payloads as [$subscriptionId, $data]) {
+            $updates[] = new Update(
+                $this->graphQlMercureSubscriptionUriGenerator->generateMercureTopicUrl($subscriptionId),
+                $this->serializer->serialize($data, key($this->formats)),


@alanpoulain Wouldn't this serialize the messages using the same format (defaults to JSON-LD I believe?) as the normal mercure updates? I think it would make sense to use GraphQL here, no?

The normalized data is sent as JSON like the GraphQL endpoint would do. Maybe I should use json instead of the key of formats, WDYT?

I've changed it to mimic how the endpoint is doing it:

core/src/GraphQl/Action/EntrypointAction.php

Line 85 in 9298c45

return new JsonResponse($executionResult->toArray($this->debug));

.

dunglas · 2020-01-06T20:47:43Z

src/Bridge/Doctrine/EventListener/PublishMercureUpdatesListener.php


    /**
     * @param array<string, string[]|string> $formats
     */
-    public function __construct(ResourceClassResolverInterface $resourceClassResolver, IriConverterInterface $iriConverter, ResourceMetadataFactoryInterface $resourceMetadataFactory, SerializerInterface $serializer, array $formats, MessageBusInterface $messageBus = null, callable $publisher = null, ExpressionLanguage $expressionLanguage = null)
+    public function __construct(ResourceClassResolverInterface $resourceClassResolver, IriConverterInterface $iriConverter, ResourceMetadataFactoryInterface $resourceMetadataFactory, SerializerInterface $serializer, array $formats, MessageBusInterface $messageBus = null, callable $publisher = null, ?GraphQlSubscriptionManagerInterface $graphQlSubscriptionManager = null, ?GraphQlMercureSubscriptionUriGeneratorInterface $graphQlMercureSubscriptionUriGenerator = null, ExpressionLanguage $expressionLanguage = null)


The position of $expressionLanguage should be kept to prevent a BC break.

The expression language is not defined in the configuration so it would mean doing dirty tricks.
Since it's experimental, I thought it would be better to not prevent the BC.

dunglas · 2020-01-06T20:50:57Z

src/Bridge/Doctrine/EventListener/PublishMercureUpdatesListener.php

+     *
+     * @return Update[]
+     */
+    private function getGraphQlSubscriptionUpdates($object, array $targets, string $type): array


Couldn't we move the GraphQL-specific code in a decorator?

It would mean introducing methods to get the created/updated/deleted items and to duplicate the code in postFlush and a little from publishUpdate. Is it worth it?

dunglas · 2020-01-06T20:54:18Z

src/GraphQl/Resolver/Factory/ItemSubscriptionResolverFactory.php

+            $resourceMetadata = $this->resourceMetadataFactory->create($resourceClass);
+
+            if ($subscriptionId && $resourceMetadata->getAttribute('mercure', false)) {
+                if (!$this->mercureSubscriptionUriGenerator) {


Shouldn't this exception be thrown in the constructor?

I've thought the subscription system to be independent of Mercure. Here, we need to see if the resource has a mercure attribute and, if it's the case, throw the exception if the bundle is not required. We cannot do this in the constructor.

dunglas · 2020-01-06T20:59:18Z

src/GraphQl/Subscription/MercureSubscriptionUriGenerator.php

+    private $requestContext;
+    private $hub;
+
+    public function __construct(RequestContext $requestContext, string $hub)


Maybe could we inject the values of the router.request_context.* parameters instead (https://symfonycasts.com/screencast/mailer/route-context#setting-router-request-context). It will avoid a coupling with Symfony.

It would be nice but these parameters are not dynamically updated whereas it's the case for the service.

dunglas · 2020-01-06T21:00:20Z

src/GraphQl/Subscription/MercureSubscriptionUriGenerator.php

+    public function generateMercureTopicUrl(string $subscriptionId): string
+    {
+        if (('' === $scheme = $this->requestContext->getScheme()) || ('' === $host = $this->requestContext->getHost())) {
+            throw new \RuntimeException('Cannot generate topic URL from request context.');


I suggest to use a default value if the context isn't available, such as https://api-platform.com/subsriptions/. IRIs don't have to exist.

OK, it's an edge case anyway.

Actually, maybe should we always use this default, and let the user pass another namespace if required, instead of relying on the router.

dunglas · 2020-01-06T21:02:07Z

src/GraphQl/Subscription/MercureSubscriptionUriGeneratorInterface.php

+ *
+ * @author Alan Poulain <contact@alanpoulain.eu>
+ */
+interface MercureSubscriptionUriGeneratorInterface


MercureSubscriptionIriGeneratorInterface for consistency (we use IRI everywhere).

dunglas · 2020-01-06T21:02:34Z

src/GraphQl/Subscription/MercureSubscriptionUriGeneratorInterface.php

+
+    public function generateMercureTopicUrl(string $subscriptionId): string;
+
+    public function generateMercureUrl(string $subscriptionId): string;


I suggest to simplify this interface and to provide only a single method getTopic(string $subscriptionId)

I can remove the generateMercureTopicIri but I need two methods: one for getting the topic IRI and one for getting the Mercure URL (with the topic in the query).

dunglas · 2020-01-06T21:03:51Z

src/GraphQl/Subscription/SubscriptionIdentifierGenerator.php

+ *
+ * @author Alan Poulain <contact@alanpoulain.eu>
+ */
+final class SubscriptionIdentifierGenerator implements SubscriptionIdentifierGeneratorInterface


Are this class and the related really necessary? Cannot we merge it with the topic generator, and use the topic as ID everywhere?

This class is necessary because I use a random string for the subscriptions. I can remove it if I use a hash instead. Maybe using a SHA-256 hash would be a better idea like we discussed since Apollo is doing it for its persisted queries.
Also I cannot add it to the generator since it's not related to Mercure.

Using a hash sounds like a good idea!

Done. I think keeping the service is a good idea since it allows to change how the identifier is generated if needed.

lukasluecke · 2020-01-23T08:23:11Z

@alanpoulain I have now implemented this in my project with a matching frontend client, and it works great! Thank you very much for your work! 🏆

I do have some questions / improvement requests for the future:
Currently the subscriptions are bound only to the update of the main resource (the one that generates the subscription operation, based on mercure=true), right?
Is there any way we could update this also based on the selection set, so inner resource updates would also trigger new data (would something like #3206 help)?
Could we add custom Subscriptions, similar to the Queries and Mutations, later on? Not sure how exactly their signature would look like, but that could be an interesting feature - either mix multiple resources and their updates in one subscription, or even integrate "outside" sources, e.g. custom Messenger messages.

alanpoulain · 2020-01-23T10:01:55Z

Thank you for your feedback @lukasluecke!
For your first question, yes #3206 would help to do this.
Custom subscriptions could be a nice addition later on 🙂

lukasluecke · 2020-01-24T10:01:50Z

@alanpoulain Great! After some more testing I think it would probably be good to get #3206 (or something comparable) in together with this, otherwise things might get confusing 🙈 Could you maybe take a look at this?

alanpoulain · 2020-01-24T10:03:08Z

Can't it be done later?

lukasluecke · 2020-01-24T10:13:14Z

Of course, it's not strictly required / related to the changes directly 🙂 I'll try to look into it in the next days 👍

phtmgt · 2020-05-05T07:34:52Z

Hey guys, is this now widely available? Doesn't seem to work on 2.5.4 (I get 'Internal Error: Subscription type is not defined the Schema' with relay). Is this my fault or it's not yet implemented in this version? Thanks.

alanpoulain · 2020-05-05T07:52:32Z

Not yet. In 2.6.

lukasluecke · 2020-09-01T10:07:25Z

@alanpoulain Quick question - is there an easy solution to make Mercure use the GraphQL serializer & format for the updates it sends? Would make it a lot easier to properly integrate clientside, instead of dealing with the JSONLD and the differences.

alanpoulain · 2020-09-01T12:30:22Z

It shouldn't use JSON-LD by default but the GraphQL format. See this test:

core/features/graphql/subscription.feature

Lines 196 to 207 in 2c87089

    
               Then the following Mercure update with topics "http://example.com/subscriptions/[a-f0-9]+" should have been sent: 
        
               """ 
        
               { 
        
                   "dummyMercure": { 
        
                       "id": 1, 
        
                       "name": "Dummy Mercure #1 updated", 
        
                       "relatedDummy": { 
        
                           "name": "RelatedDummy #1" 
        
                       } 
        
                   } 
        
               } 
        
               """

lukasluecke · 2020-09-01T12:57:49Z

@alanpoulain Sorry, I was a little confused when posting. It's not JSONLD, but it's also not what usually is available in a GraphQL response.

I'm using a normalized cache in the frontend, and in order to match the subscription results I need the __typename and id to match - however in Mercure updates the __typename is not available at all, and the id is just the numeric ID, not the IRI as it should be. Is this just a misconception on my side?

I fixed it temporarily in SubscriptionManager with the following lines (not nice, but it works 😅):

$rootObject = array_keys($data)[0];
$data[$rootObject]['__typename'] = $this->resourceMetadataFactory->create($resourceClass)->getShortName();
$data[$rootObject]['id'] = $iri;

alanpoulain · 2020-09-01T13:17:00Z

I understand for the __typename, because it is given by graphql-php and not the serialize stage.

For the ID and not the IRI, I don't remember why it's like this (and I even wrote it in the tests), but it seems an issue to me indeed. Maybe the context is not given correctly to the stage.

SlimenTN · 2020-11-19T15:02:32Z

hey @alanpoulain thanks for your great work to integrate graphql subscription
but I'm using angular with apiplatform and when I try to create a subscription I get this error

Schema is not configured for subscriptions.

I want to get notification whenever a new one is created so I have a Notification entity with mercure activated mercure=true, and this is my Angular side code:

ngOnInit(){
    this.apollo.subscribe({
      query: gql`
        subscription{
          newNotification{
            title
            content
          }
        }`,
      fetchPolicy: "no-cache"
    })
    .subscribe(res => console.log(res));
  }

why it doesn't know the subscription schema ?
same thing happens if I run it in the Qraphiql IDE that comes with ApiPlatform:

lukasluecke · 2020-11-19T18:32:02Z

You need to use a custom link that knows how to handle the mercure subscription. I can give you an example of what I'm using tomorrow, but it's for urql, not Apollo.

SlimenTN · 2020-11-19T19:08:13Z

@lukasluecke ok thanks for your response
this custom link is it mentioned anywhere in the documentation? because there isn't a lot mentioned about the subscription in apiplatform's documentation!

xleliberty · 2021-02-24T08:39:33Z

Hi, @lukasluecke @SlimenTN, i'm trying using apollo client with a splitLink to connect to api-platform subscriptions, but i didn't succeeded and didn't find any good exemple code.

Have some of you found a "good" way to implement this ?
@lukasluecke would you mind sharing your urql code ?

Thanks
Xavier

SlimenTN · 2021-02-24T08:53:04Z

Hi @xleliberty actually still waiting for @lukasluecke's example, because as I mentioned there isn't a lot mentioned about the subscription in apiplatform's documentation.
So I hope he can answer us soon 😄

xleliberty · 2021-02-24T09:24:00Z

Ok Thanks Slimen,

You're right, i feel too like we are missing a bit of informations about subscription consumption on the JS side. Once we have more info bits, i'll try to add a little doc PR for this.

@alanpoulain , maybe you have some hints about js consumption ?

Thanks

lukasluecke · 2021-02-26T10:57:28Z

@SlimenTN @xleliberty Sorry, totally forgot about this! You can see my example at https://gist.github.com/lukasluecke/0c8e2c12a955198926779a8691874c03.

xleliberty · 2021-03-03T11:05:18Z

Hi @lukasluecke , thanks for your input !

lermontex · 2021-06-18T00:10:47Z

@SlimenTN, @xleliberty with Apollo, you can use it like a normal Query, something like this:

    const { result } = useQuery(
      gql`
        subscription yourSubscriptionName($id: ID!) {
          updateTaskSubscribe(input: { id: $id }) {
            dummyMercure {
              id
              name
              relatedDummy {
                name
              }
            }
            mercureUrl
          }
        }
      `,
      () => ({
        id: "/dummy_mercures/1",
      })
    );

phtmgt · 2022-02-22T08:39:27Z

Anyone tried implementing this with react-relay on the frontend?

alanpoulain added the GraphQL label Dec 20, 2019

alanpoulain force-pushed the graphql-subscription branch from e590ae1 to 6ca6c2c Compare December 21, 2019 13:50

lukasluecke reviewed Jan 6, 2020

View reviewed changes

alanpoulain force-pushed the graphql-subscription branch 3 times, most recently from 9197b5c to 11bfab4 Compare January 6, 2020 16:13

dunglas requested changes Jan 6, 2020

View reviewed changes

alanpoulain force-pushed the graphql-subscription branch from 11bfab4 to b49b30c Compare January 7, 2020 15:40

alanpoulain requested a review from dunglas January 7, 2020 15:41

alanpoulain force-pushed the graphql-subscription branch from cc9da4f to 51a0581 Compare January 11, 2020 15:44

alanpoulain mentioned this pull request Jan 15, 2020

subscription implementation webonyx/graphql-php#9

Closed

lukasluecke mentioned this pull request Jan 23, 2020

GraphQl: interface implementation #3306

Closed

alanpoulain added 3 commits February 3, 2020 11:11

Subscription support

ee5bdcb

Use sha256 hash for generating subscription identifiers

2221eb3

Update CHANGELOG

095ea9b

alanpoulain force-pushed the graphql-subscription branch from 51a0581 to 095ea9b Compare February 3, 2020 10:31

alanpoulain merged commit d6f2603 into api-platform:master Feb 3, 2020

lermontex mentioned this pull request Jun 9, 2021

Create a Mercure adapter for GraphQL subscriptions (Apollo) dunglas/mercure#449

Open

lermontex mentioned this pull request Jun 17, 2021

[GraphQL] Subscribing to a resource with a secure resource update method #4330

Closed

oojacoboo mentioned this pull request Oct 7, 2023

Integration of subscriptions thecodingmachine/graphqlite#13

Open


		public function generateMercureTopicUrl(string $subscriptionId): string;

		public function generateMercureUrl(string $subscriptionId): string;

[GraphQL] Mercure Subscription Support #3321

[GraphQL] Mercure Subscription Support #3321

Conversation

alanpoulain commented Dec 20, 2019 • edited Loading

alanpoulain commented Dec 21, 2019

lukasluecke commented Dec 21, 2019

CvekCoding commented Dec 21, 2019

lukasluecke commented Jan 6, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alanpoulain Jan 6, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukasluecke commented Jan 23, 2020

alanpoulain commented Jan 23, 2020

lukasluecke commented Jan 24, 2020 • edited Loading

alanpoulain commented Jan 24, 2020

lukasluecke commented Jan 24, 2020

phtmgt commented May 5, 2020 • edited Loading

alanpoulain commented May 5, 2020

lukasluecke commented Sep 1, 2020

alanpoulain commented Sep 1, 2020

lukasluecke commented Sep 1, 2020

alanpoulain commented Sep 1, 2020

SlimenTN commented Nov 19, 2020

lukasluecke commented Nov 19, 2020

SlimenTN commented Nov 19, 2020

xleliberty commented Feb 24, 2021

SlimenTN commented Feb 24, 2021

xleliberty commented Feb 24, 2021

lukasluecke commented Feb 26, 2021

xleliberty commented Mar 3, 2021

lermontex commented Jun 18, 2021

phtmgt commented Feb 22, 2022

alanpoulain commented Dec 20, 2019 •

edited

Loading

alanpoulain Jan 6, 2020 •

edited

Loading

lukasluecke commented Jan 24, 2020 •

edited

Loading

phtmgt commented May 5, 2020 •

edited

Loading