Custom SQS queue reader for Laravel

The Laravel SQS Queue Reader is a powerful extension designed to seamlessly integrate external webhooks into your Laravel application. By leveraging the reliability and scalability of Amazon Simple Queue Service (SQS), this extension ensures that your application efficiently processes incoming webhooks, minimizing downtime and enhancing overall performance.

Key Features:

Effortless Webhook Integration:

Easily integrate external webhooks into your Laravel application without compromising on performance.

Queue-Based Processing:

Harness the power of Amazon SQS to queue incoming webhooks, allowing for asynchronous and parallel processing, ensuring optimal response times.

Reliability and Scalability:

SQS provides a robust and scalable infrastructure, ensuring that your application can handle varying webhook loads without compromising on stability.

Seamless Laravel Integration:

Designed as a Laravel extension, the Webhook Queue Reader seamlessly integrates into your Laravel project, following Laravel's coding standards and conventions.

Configurable Settings:

Customize the extension's settings to align with your application's requirements, including queue names, visibility timeout, and other SQS-specific configurations.

Detailed Logging:

Gain insights into the webhook processing flow with detailed logging, helping you troubleshoot and monitor the system effectively.

How It Works:

Webhook Registration:

Register external webhooks with your Laravel application by providing the webhook URL.

SQS Queue Integration:

Incoming webhooks are efficiently processed through the SQS queue, ensuring optimal handling of webhook payloads.

Asynchronous Processing:

Leverage the asynchronous processing capabilities of SQS to handle webhooks in the background, preventing any impact on your application's response times.

Automatic Retries:

Benefit from SQS's automatic retries, ensuring that failed webhook processing attempts are retried without manual intervention.

Custom SQS queue reader for Laravel projects that supports raw JSON payloads and reads multiple messages. Laravel expects SQS messages to be generated in a specific format that includes job handler class and a serialized job.

Note: Implemented to read multiple messages from queue.

This library is very useful when you want to parse messages from 3rd party applications such as stripe webhooks, shopify webhooks, mailgun web hooks, custom JSON messages and so on.

Getting Started

Install Custom SQS queue reader for Laravel via composer:

composer require palpalani/laravel-sqs-queue-json-reader

You can publish the config file and Configure your SQS settings in the Laravel configuration file.

php artisan vendor:publish --provider="palPalani\SqsQueueReader\SqsQueueReaderServiceProvider" --tag="config"

This is the contents of the published config file:

/**
 * List of plain SQS queues and their corresponding handling classes
 */
return [

    // Separate queue handler with corresponding queue name as key.
    'handlers' => [
        'stripe-webhooks' => [
            'class' => App\Jobs\StripeHandler::class,
            'count' => 10,
        ],
        'mailgun-webhooks' => [
            'class' => App\Jobs\MailgunHandler::class,
            'count' => 10,
        ]
    ],

    // If no handlers specified then default handler will be executed.
    'default-handler' => [

        // Name of the handler class
        'class' => App\Jobs\SqsHandler::class,

        // Number of messages need to read from SQS.
        'count' => 1,
    ]
];

If the queue is not found in 'handlers' array, SQS payload is passed to default handler.

Register your webhooks with your Laravel application.

Add sqs-json connection to your config/queue.php, Example:

    [
        // Add new SQS connection
        'sqs-json' => [
            'driver' => 'sqs-json',
            'key'    => env('AWS_ACCESS_KEY_ID', ''),
            'secret' => env('AWS_SECRET_ACCESS_KEY', ''),
            'prefix' => env('AWS_SQS_PREFIX', 'https://sqs.us-west-2.amazonaws.com/1234567890'),
            'queue'  => env('AWS_SQS_QUEUE', 'external-webhooks'),
            'region' => env('AWS_DEFAULT_REGION', 'us-west-2'),
        ],
    ]

In your .env file, choose sqs-json as your new default queue driver:

QUEUE_DRIVER=sqs-json

Enjoy seamless, reliable, and scalable webhook processing!

Dispatching to SQS

If you plan to push plain messages from Laravel, you can rely on DispatcherJob:

use palPalani\SqsQueueReader\Jobs\DispatcherJob;

class ExampleController extends Controller
{
    public function index()
    {
        // Dispatch job with some data.
        $job = new DispatcherJob([
            'music' => 'Ponni nathi from PS-1',
            'singer' => 'AR. Rahman',
            'time' => time()
        ]);

        // Dispatch the job as you normally would
        // By default, your data will be encapsulated in 'data' and 'job' field will be added
        $this->dispatch($job);

        // If you wish to submit a true plain JSON, add setPlain()
        $this->dispatch($job->setPlain());
    }
}

Above code will push the following JSON object to SQS queue:

{"job":"App\\Jobs\\SqsHandler@handle","data":{"music":"Sample SQS message","singer":"AR. Rahman","time":1464511672}}

'job' field is not used, actually. It's just kept for compatibility with Laravel Framework.

Processing job

Run the following commnd for testing the dispatched job.

php artisan queue:work sqs-json

For production, use supervisor with the following configuration.

[program:sqs-json-reader]
process_name=%(program_name)s_%(process_num)02d
command=php /var/html/app/artisan queue:work sqs-json --sleep=60 --timeout=10 --tries=2 --memory=128 --daemon
directory=/var/html/app
autostart=true
autorestart=true
startretries=10
user=root
numprocs=1
redirect_stderr=true
stdout_logfile=/var/html/app/horizon.log
stderr_logfile=/tmp/horizon-error.log
stopwaitsecs=3600
priority=1000

If you are using multiple connection, then duplicate above supervisor configutation and change the connection name.

Receiving from SQS

If a 3rd-party application or API Gateway to SQS implementation is creating custom-format JSON messages, just add a handler in the config file and implement a handler class as follows:

use Illuminate\Contracts\Queue\Job as LaravelJob;

class SqsHandlerJob extends Job
{
    /**
     * @var null|array $data
     */
    protected $data;

    /**
     * @param LaravelJob $job
     * @param null|array $data
     */
    public function handle(LaravelJob $job, ?array $data): void
    {
        // This is incoming JSON payload, already decoded to an array
        var_dump($data);

        // Raw JSON payload from SQS, if necessary
        var_dump($job->getRawBody());
    }
}

Note:

Ensure that your Laravel application is configured with the necessary AWS credentials and permissions to interact with SQS.

Enhance your Laravel application's webhook processing capabilities with the Laravel Webhook Queue Reader. Efficient, reliable, and designed for optimal performance!

For more information about AWS SQS check offical docs.

Testing

We already configured the script, just run the command:

composer test

For test coverage format, run the command:

composer test-coverage

For code analyse, run the command:

composer analyse

For code format, run the command:

composer format

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

If you want to contribute, then you may want to test it in a real Laravel project:

• Fork this repository to your GitHub account.
• Create a Laravel app locally.
• Clone your fork in your Laravel app's root directory.
• In the /laravel-sqs-queue-json-reader directory, create a branch for your fix, e.g. feature/awesome-feature.

Install the packages in your app's composer.json:

{
    // ...
    "require": {
        "palpalani/laravel-sqs-queue-json-reader": "*",
    },
    "minimum-stability": "dev",
    "repositories": [
        {
            "type": "path",
            "url": "path/to/location"
        }
    ],
    // ...
}

Now, run composer update.

Other Laravel packages

GrumPHP rector task GrumPHP with a task that runs RectorPHP for your Laravel projects.

Email Deny List (blacklist) Check - IP Deny List (blacklist) Check Deny list (blacklist) checker will test a mail server IP address against over 50 DNS based email blacklists. (Commonly called Realtime blacklist, DNSBL or RBL).

Spamassassin spam score of emails Checks the spam score of email contents using spamassassin database.

Laravel Login Notifications A login event notification for Laravel projects. By default, it will send notification only on production environment only.

Laravel Toastr Implements toastr.js for Laravel. Toastr.js is a Javascript library for non-blocking notifications.

Beast Beast is Screenshot as a Service using Nodejs, Chrome and Aws Lamda. Convert a webpage to an image using headless Chrome Takes screenshot of any given URL/Html content and returns base64 encoded buffer.

eCommerce Product Recommendations Analyse order history of customers and recommend products for new customers which enables higher sales volume.

Security Vulnerabilities

Please review our security policy on how to report security vulnerabilities.

Credits

• palPalani
• All Contributors

Need Help?

If you spot a bug or have a question or feature request, please submit a detailed issue, and wait for assistance.

License

The MIT License (MIT). Please see License File for more information.