images in subfolders doesn't work

[raaone7]

you may try this yourself to reproduce

I tried this solution by putting test.jpg in root of the bucket - works perfect
eg. test123.com/test.jpg is the path location
test123.com/300x300/smart/diwali.jpg - works perfect

try the same image test.jpg in a subfolder of the same bucket - doesn't work, says keys not present
eg. test123.com/folder1/folder2/test.jpg is the path location
test123.com/300x300/smart/folder1/folder2/test.jpg - doesn't work

error in cloud watch

2019-06-23T09:05:46.801Z	f8642cb7-0584-4b30-9313-27aee17205c0	ImageRequest {
requestType: 'Thumbor',
bucket: 'xxx.xxxx.x.xxx',
key: 'test.jpg',
edits: { resize: { width: 300, height: 300 } },
originalImage: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 01 2c 01 2c 00 00 ff e1 00 d7 45 78 69 66 00 00 49 49 2a 00 08 00 00 00 01 00 0e 01 02 00 b5 00 00 00 1a 00 ... > }

looks like while retrieving the key:

    parseImageKey(event, requestType) {
        if (requestType === "Default") {
            // Decode the image request and return the image key
            const decoded = this.decodeRequest(event);
            return decoded.key;
        } else if (requestType === "Thumbor" || requestType === "Custom") {
            // Parse the key from the end of the path
            const key = (event["path"]).split("/");
            return key[key.length - 1];
        } else {
            // Return an error for all other conditions
            throw ({
                status: 400,
                code: 'ImageEdits::CannotFindImage',
                message: 'The image you specified could not be found. Please check your request syntax as well as the bucket you specified to ensure it exists.'
            });
        }
    }

it should fetch based on /smart/ context root instead of default / . what do you think ?

[MartijnKooij]

+1 We are experiencing the same issue. It should not assume return key[key.length - 1]; to be the key but for thumbor requests strip the known parameters? Or otherwise at least let us configure a base path, in our case all images are located in the same subfolder so that might work. For others it may not if they have varying image folders...

[hayesry]

Hey @raaone7 and @MartijnKooij: Thanks for your input on this, since this seems to be a popular feature request (working with subfolders using the Thumbor request format), I've gone ahead and added this to our backlog for a future release. In the meantime, try giving the new JSON-based request format a try, since you can specify subfolders within the "key" property similar to how an s3.getObject({}) request would work. This may provide a workaround for you guys in the meantime while we work on this.

[zeeshanchicsol]

dear can you please fix this issue?

[jasonobrown]

Yes, please put this high on the priority list!

[raaone7]

@hayesry thanks for the acknowledgement. can you please advice us of an ETA when this issue would be fixed, and how would we update the existing setup with the fix you deliver. i suppose it would be delivered as a CFN update.

[raaone7]

@hayesry

In the meantime, try giving the new JSON-based request format a try, since you can specify subfolders within the "key" property similar to how an s3.getObject({}) request would work.

but how, in the solution been deployed, i don't do anything, it adds the S3 bucket trigger to call the lambda and send the key automatically, i don't think so we have control to alter the key in JSON, unless we over ride your code, which i can, but i prefer not to and keep it native, since the change if i do, it would not be managed support in that case, and i would need to manage the code myself.

[afolabiabass]

I am having similar issue, but strangely, I have one or two subfolder images that show, while the rest don't. Getting the error {"status":500,"code":"NoSuchKey","message":"The specified key does not exist."}. What is the solution for this?

[mybluedog24]

Same here, just found out it's an issue related to the path. Really need this feature, but going to use JSON request format for now.

[zeeshanchicsol]

Can anyone please fix this issue i am not able to deploy this solution.

[soupy1976]

Due to the fact that AWS have updated the lambda execution environment, which caused our old thumbor implementation to break, we had to implement this ourselves. If anyone is interested, this is the code we have so far (this is not tested in production yet, so no guarantees...but maybe helpful to someone)

PLEASE NOTE: This was written in a way that works with the way that our application generates the Thumbor urls. It makes assumptions about the way the urls are constructed. It assumes that the dimensions are in the request somewhere, and does not handle all Thumbor options. It also assumes that anything it doesn't recognise after the image dimensions param is the start of the path, and then it takes the rest of the path from there. You will most likely not be able to use this without adapting it at least to some extent.

    /**
    * Gets the S3 key of the file from the request path
    * Please note: this function assumes that the dimensions will be in the request somewhere (eg /200x300/ or whatever)
    * @param {Object} event - The request body.
    */
    getFileS3Key(event) {
        this.path = event.path;
        const pathParts = this.path.split('/');
        var fileS3Key = '', sizeParameterFound = false;
        const matchSize = new RegExp(/^\d+x\d+$/);

        for (var i = (securityHashPathIndex + 1); i < pathParts.length; i++) {
            const pathPart = pathParts[i];
            if (fileS3Key.length > 0) {
                fileS3Key += '/';
            }
            else if (!sizeParameterFound && matchSize.test(pathPart)) {
                sizeParameterFound = true;
                continue;
            }
            else if (pathPart === ('fit-in') || pathPart.includes('filters:') || !sizeParameterFound) {
                continue;
            }

            fileS3Key += pathPart;
        }
        return fileS3Key;
    }

securityHashPathIndex is 1 (in my case at least).

and a unit test for that:

// ----------------------------------------------------------------------------
// getFileS3Key()
// ----------------------------------------------------------------------------
describe('getFileS3Key()', function () {
    describe('001/fileS3Key', function () {
        it(`Should pass if the file S3 key is being extracted correctly from the url path`, function () {
                // Arrange
                const event = {
                    path: "/somesecurityhash/fit-in/200x300/filters:grayscale()/some/filexfile/path/blah/test-image-001.jpg"
                }
                // Act
                const thumborMapping = new ThumborMapping();
                var result = thumborMapping.getFileS3Key(event);
                // Assert
                const expectedResult = "some/filexfile/path/blah/test-image-001.jpg";
                assert.deepEqual(result, expectedResult);
            });
    });
});

[dyarbrough]

This solution is unusable for us without the ability to use subfolders.