How do I implement Memory Storage

[sandinosaso]

I am using memory for users_credentials and access_token I am using grant_type 'password' I am succesfull validating user with memory saved user_credential and token the problem is when getting a new token it is not saved to memory again do not know how to implement this kind of funcionality

What I am doing is:

$tokens_in_memory = array('access_tokens'=>array(                                                       '42e13ba1c0e33036889617ee45c0d0ee7b28653f'=>array('access_token'=>'42e13ba1c0e33036889617ee45c0d0ee7b28653f',                                                                                                         'client_id'   => 'test_client',                                                                                                         'user_id'     => '_app_user',                                                                                                       'expires'     => strtotime('2014-10-05 22:45:48'))
                                                        )
                                 );

$tokenStorage = new OAuth2\Storage\Memory($tokens_in_memory);
...
...
$this->server->addStorage($tokenStorage, 'access_token');

This is done on my controller init method I think it is overwriting everytime so I am not able to generate new tokens in memory, the PDO solution works geat but doing some kind of apache benchmark it shows that is 100x slower than doing in memory, I will be checking many tokens per second so the fastest I can do it the better, it would be great to suppor MemCache as well I will investigate if I can do it and make a pull request.

I inspected code and see that TokenController calls:

function grantAccessToken(RequestInterface $request, ResponseInterface $response)

and it returns:

return $grantType->createAccessToken($this->accessToken, $clientId, $grantType->getUserId(), $requestedScope);

and at the end in te createAccessToken it calls:

 $this->tokenStorage->setAccessToken($token["access_token"], $client_id, $user_id, $this->config['access_lifetime'] ? time() + $this->config['access_lifetime'] : null, $scope);

In my case tokenStorage is memory but I do not see news token persisted, maybe you have a php server in memory running, or do you save to session or something

I am doing an api using Yii is kind of easy integrate it with your library and it would be great to have it merged next releases of Yii if I make it work I will share code to make this library work with Yii .

Thank you, you all

[bshaffer]

Hey there,
The MemoryStorage is exactly what it sounds like - the items are stored in memory and in memory alone. This means changes to this are not persisted between requests.

This is useful for setting up clients quickly and for running tests - you can load clients/access tokens from YAML or static PHP into the Memory storage class. But any changes made have to be added on every single request, as these changes are never persisted to subsequent requests.

You will want to use PDO storage to create new access tokens.

[sandinosaso]

That is fine I understand now. I Copy the PDO Storage class and made a new one PDOMemCache, this class use Memcache for caching in memory (MemCache is super fast) so when user is asking for a resource in the getAccessToken method I run the query against the database only first time if I have not in memcache, I save the result for future use, let say you got a memcache 10 min its ok. This way I avoid the overhead of making a sql query to the database each time, think about an api writted for an phone app and that a phone app calls many time to the api with the same token, this multiplies by the numbers of user using the app it a huge number, I am implementing an api that will be used widely by android, iphone, windows phone it must by really fast.

I do some apache benchmark for some service I provide and time is really important, for example
making 1000 request 100 per second when using memory directly it takes 2.757 seconds and when using PDO against mysql same requests takes 10.935 seconds, that is not affordable for me, now using memcache it takes 3.206 seconds that pretty good. I would like to make a pull request if you are interested in this kind of ideas, currently I modified only one function but this aplies for every query PDO makes, maybe it would be a new Storage PDOMemCache or PDO with an option of using memcache or not

This is what I did:

/* OAuth2\Storage\AccessTokenInterface */
    public function getAccessToken($access_token)
    {
        $stmt = $this->db->prepare(sprintf('SELECT * from %s where access_token = :access_token', $this->config['access_token_table']));

        $cacheKey = 'query-'.md5(serialize(array($stmt->queryString, compact('access_token'))));           

        # Try and get from memory
        $checkCache = $this->memcache->get($cacheKey);

        # We have some data
        if(!empty($checkCache)) {
            return $checkCache;
        }else{
            $token = $stmt->execute(compact('access_token'));
            if ($token = $stmt->fetch()) {
                // convert date string back to timestamp
                $token['expires'] = strtotime($token['expires']);
                $this->memcache->set($cacheKey, $token, 0, 600);
            }
        }

        return $token;
    }

[bshaffer]

This is a great idea!

What you could actually do is make a memcache class that accepts any other storage class as its first argument, and calls that class if the cache key doesn't exist. It would look something like this:

class Memcache implements AccessTokenInterface
{
    protected $storage;
    protected $memcache;

    public function __construct(AccessTokenInterface $storage, $memcache)
    {
        $this->storage = $storage;
        $this->memcache = $memcache;
    }

    public function getAccessToken($access_token)
    {
        $cacheKey = 'storage-'.$access_token;           

        # Try and get from memory
        $accessToken = $this->memcache->get($cacheKey);

        # We have some data
        if(!empty($accessToken)) {
            return $accessToken;
        }

        $accessToken = $this->storage->getAccessToken('access_token');
        $this->memcache->set($cacheKey, $accessToken, 0, 600);

        return $token;
    }
}

[sandinosaso]

WooooW that would be really helpfull can this be added to the Main repo,
And the only other function it must implement is
public function setAccessToken($oauth_token, $client_id, $user_id, $expires, $scope = null);

I think this method will always have to update the token in the passed $storage and also in mem_cache ?..
The only requeriment for this to work is having mem_cache enabled in php so I think is a great solution for making apps calls for resources super fast, maybe de 600 seconds time may be a parameter as well as you may need to choose how much your memcache is valid or can use the same time you used for accessToken.
The only thing you have to add is a first line to include Memcache
like use Memcache

What do you think? I can implement it and share some benchmark showing it benefits, what I must do to get a pull request approved by you?

Regards.
Sandino.

[bshaffer]

I would love that very much! Please do so.

[sandinosaso]

I have done it, I need your help with the test, I run phpunit and get the error:

The data provider specified for OAuth2\Storage\AccessTokenTest::testSetAccessToken is invalid.
could not find driver

For every test, what I messed up, is something that needs to be configured before running test?, Is there any documentation on running tests somewhere?.

Regards.
Sandino

[bshaffer]

Hmm... I've written the tests so it should skip any tests that you don't have drivers for. Can you run the tests with phpunit -v and see if there is any more information? Feel free to post the full output of the tests here.

[bshaffer]

Also, if you post your branch on github I can pull it down and run the tests, as my machine is set up. To run ALL the tests, you need the following:

• predis/predis
• thobbs/phpcassa
• aws/aws-sdk-php
• MongoDB php extension
• Couchbase php extension
• Redis/Mongo/Cassandra databases running locally

But as i said, the library should skip those tests if they are not configured.

[pjebs]

I was wondering what the progress was of this? It would definitely be a useful addition

[bshaffer]

I don't think @sandinosaso ever submitted a pull request... but feel free to submit one as well @pjebs

[sandinosaso]

I never did but I can if you let me, the code is really simple as bshaffer suggested I implemented passing a valid storage, so I am using PDO storage with a mysql database and usign Memcache to faster retrieve access tokens

namespace OAuth2\Storage;
use Memcache;

class MemCacheToken implements AccessTokenInterface
{
    protected $storage;
    protected $memcache;

    public function __construct(AccessTokenInterface $storage, $config=array())
    {
        $this->storage = $storage;

        $this->memcache = new Memcache;

        $host    = isset($config['host'])    ? $config['host']    : 'localhost';
        $port    = isset($config['port'])    ? $config['port']    : 11211;
        $timeout = isset($config['timeout']) ? $config['timeout'] : 1;

        $this->memcache->connect($host, $port, $timeout);
    }

    public function getAccessToken($access_token)
    {
        $cacheKey = 'storage-'.$access_token;           

        # Try and get from memory
        $accessToken = $this->memcache->get($cacheKey);

        # We have some data
        if(!empty($accessToken)) {
            return $accessToken;
        }

        $accessToken = $this->storage->getAccessToken('access_token');
        $this->memcache->set($cacheKey, $accessToken, 0, strtotime($accessToken['expires']));

        return $accessToken;
    }

    public function setAccessToken($oauth_token, $client_id, $user_id, $expires, $scope = null)
    {
        $cacheKey = 'storage-'.$oauth_token;           

        $this->storage->setAccessToken($oauth_token, $client_id, $user_id, $expires, $scope);
        $updatedAccessToken = $this->storage->getAccessToken($oauth_token);

        $result = $this->memcache->replace($cacheKey, $updatedAccessToken, 0, $expires);
        if( $result == false ) 
        { 
            $result = $this->memcache->set($cacheKey, $updatedAccessToken, 0, $expires);
        }

    }
}

For use it just do

        try {
            $pdotokenStorage = new OAuth2\Storage\Pdo(array('dsn' => $dsn, 'username' => $username, 'password' => $password));
        }catch(PDOException $e) {
            //Send email with error
            Yii::app()->EmailSenderComponent->send('Database Error', $e->getMessage());
                        //Log the error
            Yii::log('Database error :'.$e->getMessage(), 'error', 'application.ApiController.init');
            $this->_sendResponse(500, json_encode(array ('status'=> 500, 'message'=> 'Internal error contact the admin', 'code'=> 20003)), array('Content-type'   => 'application/json') );
        }
        // USING PDOMEMCACHE
        $tokenStorage = new OAuth2\Storage\MemCacheToken($pdotokenStorage);

You can also pass an array of parameter to choose wich MemCache server you want to connect
Like this $tokenStorage = new OAuth2\Storage\MemCacheToken($pdotokenStorage, array('host' =>'YOURHOST', 'port'=>YOUR_PORT_NUMBER));

[sandinosaso]

I created a pull request #608