Perfomance Issues

[KeKs0r]

Hi,

I am having several sites where I show a lot of user profile picture thumbs. Those sites are extremely slow. I think it is related to the imagine filter. I have created a performance test, that collects the exact same data as my controller and then accesses the exact same variables / functions as my view.

All those calculations take about 8ms, but as soon as I add the following line it takes already 2 seconds ( for 2 users, exponentially growing with number of users, because its in a loop):

$imagineCManager = $this->get('liip_imagine.cache.manager');

foreach($group->getUsers as $user){
$data[] = $imagineCManager->getBrowserPath($user->getProfilePic(), 'profile_thumb_small', false);
}

Why is that so slow? My backend is Gaufrette with AmazonS3. I thought there would be a CacheManager in between, that might cache the paths or something. Or the path itself can easily calculated from $user->getProfilePic(), because the original file is in my bucket under profile/$profilePicName and the filtered file would be under profile_thumb_small/$profilePicName. But to "calculate" this, it still takes about a second.

Also in prod the request goes through my webserver instead of just resolving the link:

<img src="/web/app.php/media/cache/profile_thumb_small/52136f2930e36.jpeg" alt="degi" class="thumbnail-small">

[havvg]

That's why there is a resolver wrapping another one caching its results: see #184

[KeKs0r]

Thanks, sometimes one should just read more carefully, I always skipped the CacheResolver, because I thought it would be replaced by the S3 resolver. Can you make a recommendation regarding the cache to use? In my environment I have APC, Memcached and redis in beta. For now I only use APC, because its just an PHP extension, but if the performance gain is sufficient, I could easily upgrade or switch to memcached.

[havvg]

We are using memcached via https://www.memcachier.com/.
I didn't test the performance differences between Apc and Memcached.

[KeKs0r]

Thanks for the valuable tips, this solved major performance issues on my site. Nevertheless I am encountering a weird behaviour, two pictures were newly added and not the final S3 URL but the URL to the media/cache controller was cached. So all pictures were loading fine except the two new ones. The only solution I found was to delete the cache entries manually. How do I prevent this from happening?

[phtmgt]

Ok, I setup the CacheResolver wraper, but we still get a huge slowdown (about 30 seconds). Basically, there is no difference in using the wrapper or s3 directly. This happens on serializing entities:

public function onPostSerialize(ObjectEvent $event)
    {
        $object = $event->getObject();
        $visitor = $event->getVisitor();
        /* @var $visitor GenericSerializationVisitor */
        $path = $this->uploaderHelper->asset($object, 'file');
        $urls = array();
        foreach (array('carouselImage', 'carouselImage2x', 'profilePhoto', 'profilePhoto2x', 'companyLogo') as $pattern) {
            $urls[$pattern] = $this->cacheManager->getBrowserPath($path, sprintf('%s', $pattern));
        }
        $visitor->addData('image_urls', $urls);
    }

Cache settings:

app.s3:
        class: Aws\S3\S3Client
        factory: [Aws\S3\S3Client, 'factory']
        arguments:
            -
                version: 'latest'
                region: 'fra1'
                endpoint: '%env(S3_BASE_URL)%'
                credentials:
                    key: '%env(S3_KEY)%'
                    secret: '%env(S3_SECRET_KEY)%'

app.imagine.cache.resolver.s3:
        class: Liip\ImagineBundle\Imagine\Cache\Resolver\AwsS3Resolver
        arguments:
            - "@app.s3"
            - '%env(S3_BUCKET_NAME)%'
        tags:
            - { name: "liip_imagine.cache.resolver", resolver: "default" }

cache.apcucache:
        class: Doctrine\Common\Cache\ApcuCache

acme.imagine.cache.resolver.s3.cache:
        class: Liip\ImagineBundle\Imagine\Cache\Resolver\CacheResolver
        arguments:
            - "@cache.apcucache"
            - "@app.imagine.cache.resolver.s3"
            - { prefix: "s3" }
        tags:
            - { name: "liip_imagine.cache.resolver", resolver: "cached_s3" }

Imagine settings:

cache: cached_s3

[apichlinski]

@plamenh Did you resolve performance issues with S3?

[phtmgt]

@apichlinski we definitely solved that, but now I don't remember how exactly. When I compare what we have right now with what I wrote a year ago, I see we are using the cacheManager differently:

$this->cacheManager->generateUrl($path, sprintf('%s', $pattern));

instead of

$this->cacheManager->getBrowserPath($path, sprintf('%s', $pattern));

See, if this helps.