Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup embedded tracker environment correctly in queuedtracking:process #17

Merged
merged 3 commits into from
Jul 24, 2015
Merged

Setup embedded tracker environment correctly in queuedtracking:process #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7cb7124
Setup embedded tracker environment in queuedtracking:process command …
Jul 22, 2015
cc9187e
Destroy tracker environment in process command before exiting command…
Jul 23, 2015
b36df9b
.travis.yml file is out of date, auto-updating .travis.yml file.
Jul 23, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,6 @@ before_install:
- '[[ "$TRAVIS_PHP_VERSION" == 5.3* ]] && export USE_ZEND_ALLOC=0 || true'

install:
- '[ ! -f ./tests/travis/install_mysql_5.6.sh ] || ./tests/travis/install_mysql_5.6.sh'

# Make sure we use Python 2.6
- '[ ! -f ./tests/travis/install_python_2.6.sh ] || ./tests/travis/install_python_2.6.sh'

# move all contents of current repo (which contains the plugin) to a new directory
- mkdir $PLUGIN_NAME
- cp -R !($PLUGIN_NAME) $PLUGIN_NAME
Expand All @@ -56,16 +51,22 @@ install:
- git clone -q https://github.com/piwik/piwik.git piwik
- cd piwik
- git fetch -q --all
- git submodule update

# make sure travis-scripts repo is latest for initial travis setup
- '[ -d ./tests/travis/.git ] || sh -c "rm -rf ./tests/travis && git clone https://github.com/piwik/travis-scripts.git ./tests/travis"'
- cd ./tests/travis ; git checkout master ; cd ../..


- export GENERATE_TRAVIS_YML_COMMAND="php ./tests/travis/generator/main.php generate:travis-yml --plugin=\"QueuedTracking\" --verbose"
- '[[ "$TRAVIS_JOB_NUMBER" != *.1 || "$TRAVIS_PULL_REQUEST" != "false" ]] || ./tests/travis/autoupdate_travis_yml.sh'

- ./tests/travis/checkout_test_against_branch.sh

- '[ ! -f ./tests/travis/install_mysql_5.6.sh ] || ./tests/travis/install_mysql_5.6.sh'

# Make sure we use Python 2.6
- '[ ! -f ./tests/travis/install_python_2.6.sh ] || ./tests/travis/install_python_2.6.sh'

- ./tests/travis/configure_git.sh

# travis now complains about this failing 9 times out of 10, so removing it
Expand Down
33 changes: 14 additions & 19 deletions Commands/Process.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,8 @@

namespace Piwik\Plugins\QueuedTracking\Commands;

use Piwik\Access;
use Piwik\Application\Environment;
use Piwik\Cache;
use Piwik\Container\StaticContainer;
use Piwik\Log;
use Piwik\Piwik;
use Piwik\Plugin;
use Piwik\Plugin\ConsoleCommand;
Expand All @@ -37,19 +35,23 @@ protected function execute(InputInterface $input, OutputInterface $output)
$systemCheck = new SystemCheck();
$systemCheck->checkRedisIsInstalled();

Access::getInstance()->setSuperUserAccess(false);
Plugin\Manager::getInstance()->setTrackerPluginsNotToLoad(array('Provider'));
$trackerEnvironment = new Environment('tracker');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this recreate the container and clear any previous created instances? Especially re the Cache? I'm not into the environment / container code so just asking :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't recreate the container, it creates a new one w/ the tracker environment. For now, this pushes a container instance onto a stack so StaticContainer::get() will use the top. Which I think means I missed something, there should probably be a $trackerEnvironment->destroy() call after processing is done, so control goes back to the CLI environment.

Note: This means there's a new Plugin\Manager and should be new Plugin instances stored in $trackerEnvironment. Since it's only done once here, shouldn't be an issue.

Eventually, we'll get rid of StaticContainer so everything is provided through DI, then we just use the environment directly and worry about nothing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this PR will fix a couple of issues.

Which I think means I missed something, there should probably be a $trackerEnvironment->destroy() call after processing is done, so control goes back to the CLI environment.

Does this mean there's more work left?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be done w/ my last commit.

$trackerEnvironment->init();

$trackerEnvironment->getContainer()->get('Piwik\Access')->setSuperUserAccess(false);
$trackerEnvironment->getContainer()->get('Piwik\Plugin\Manager')->setTrackerPluginsNotToLoad(array('Provider'));
Tracker::loadTrackerEnvironment();
$this->recreateEagerCacheInstanceWhichChangesOnceTrackerModeIsEnabled();

if (OutputInterface::VERBOSITY_VERY_VERBOSE <= $output->getVerbosity()) {
Tracker::setTrackerDebugMode(true);
$GLOBALS['PIWIK_TRACKER_DEBUG'] = true;
}

$backend = Queue\Factory::makeBackend();
$queueManager = Queue\Factory::makeQueueManager($backend);

if (!$queueManager->canAcquireMoreLocks()) {
$trackerEnvironment->destroy();

$this->writeSuccessMessage($output, array("Nothing to proccess. Already max number of workers in process."));
return;
}
Expand All @@ -63,6 +65,8 @@ protected function execute(InputInterface $input, OutputInterface $output)
}

if (!$shouldProcess) {
$trackerEnvironment->destroy();

$this->writeSuccessMessage($output, array("No queue currently needs processing"));
return;
}
Expand All @@ -76,14 +80,16 @@ protected function execute(InputInterface $input, OutputInterface $output)
$startTime = microtime(true);
$processor = new Processor($queueManager);
$processor->setNumberOfMaxBatchesToProcess(1000);
$tracker = $processor->process($queueManager);
$tracker = $processor->process();

$neededTime = (microtime(true) - $startTime);
$numRequestsTracked = $tracker->getCountOfLoggedRequests();
$requestsPerSecond = $this->getNumberOfRequestsPerSecond($numRequestsTracked, $neededTime);

Piwik::postEvent('Tracker.end');

$trackerEnvironment->destroy();

$this->writeSuccessMessage($output, array(sprintf('This worker finished queue processing with %sreq/s (%s requests in %02.2f seconds)', $requestsPerSecond, $numRequestsTracked, $neededTime)));
}

Expand All @@ -97,15 +103,4 @@ private function getNumberOfRequestsPerSecond($numRequestsTracked, $neededTimeIn

return $requestsPerSecond;
}

private function recreateEagerCacheInstanceWhichChangesOnceTrackerModeIsEnabled()
{
StaticContainer::clearContainer();
Log::unsetInstance();

$key = 'Piwik\Cache\Eager';
$container = StaticContainer::getContainer();
$container->set($key, $container->make($key));
}

}