Add BatchJobService support to beta client library.

[shakaran]

As far I read here:

https://developers.google.com/adwords/api/docs/reference/v201605/BatchJobService.BatchJob#status

"For versions newer than V201509, make a POST request to the uploadUrl and retrieve the "Location" header from the response as the url to upload operations."

Other SDKs seem have a class BatchJobUploadHelper for poll or POST the upload url.

$batch_job_service = $adWordsServices->get($session, 'BatchJobService', 'v201605', 'cm');

try
{
    /** @var BatchJobReturnValue $result */
    $result = $batch_job_service->mutate($operations);
}
catch(ApiException $e)
{
    echo $e->getMessage() . PHP_EOL;
}

if(!empty($result) && $result instanceof Google\AdsApi\AdWords\v201605\cm\BatchJobReturnValue)
{
    $batch_job = reset($result->getValue());

    // HERE IS NEEDED A POST request to $batch_job->getUploadUrl()
}
else 
{
    echo 'Result is empty or no valid';
}

$selector = new Selector();
$selector->setFields(['Status']);

$batch_jobs_page = $batch_job_service->get($selector);

var_dump($batch_jobs_page); // ALL ARE AWAITING_FILE status (BatchJobStatus::AWAITING_FILE)

if I make a direct curl POST request, I am getting a 411 error from google servers since content-length is required. For example:

curl -X POST -H 'Content-length:0' 'https://batch-job-upload-prod-1234.storage.googleapis.com/1234/1234.operations.xml?moreparams'

Even cheating the Content-length I don't get a valid response. So how should I get that the job pass to active? It would be nice a class and method like BatchJobUploadHelper::upload($batch_job)

[fiboknacky]

Hello shakaran,

I suppose you're talking about our experimental branch, right?
In that branch, we've not released the support for BatchJobService yet, so it might be not trivial if you would like to use that service currently.

On the other hand, our main branch does provide BatchJobUtils which you can use to do all necessary steps for batch processing, e.g., upload operations, download the results.

If you really need to implement batch processing by yourself using our experimental branch, please note that this is not trivial, as the XML serialization/deserialization is needed.
In addition, per this requirement you're supposed to manipulate Content-Length and Content-Range properly in order to use incremental upload.
You can have a look at our XmlSerializer and XmlDeserializer in our main branch as a reference.

For more about batch processing, please read this guide.

Best,
Knack, AdWords API Team

[shakaran]

@fiboknacky yes, I am using the experimental since new PSR0 and PSR4 is more easy to handle and write, also for integrate in other frameworks like Symfony.

I am doing a little script for add ipblocks for a adwords campaigns. And I almost have it finished, just remains change the batch job status to active after waiting file status. But I still have this issue with upload url.

As far I see all the logic in BatchJobUtils for upload is delegated to class BatchJobUtilsDelegate in the method UploadIncrementalBatchJobOperations doing a curl request. So probably I will try to dig it a bit.

Add this time, I am not even doing a batch of operations since I get a SizeLimitError.REQUEST_SIZE_LIMIT_EXCEEDED when I put more than one operation in the batch service. So my idea is only block one ip (one operation), and wait the job as DONE, after that make another batch job with one operation and so on.

Thanks for all the docs and indications about XML, I will read it and lets see if I can get something for my self. But do you have a estimated roadmap for when will be this supported? For me it is better write this new code using experimental, because probably my client will use for a long time and old lib probably is deprecated in some months or years when experimental gets more stable.

[shakaran]

@fiboknacky @vtsao I see that for implement the Batch Job upload we will need CurlUtils class or similar rewritten for experimental. Should it be included? It would be nice have a new release of experimental with this features soon, since my client is waiting only for this feature and he doesn't want develop with the old master api

[fiboknacky]

Hi @shakaran

Add this time, I am not even doing a batch of operations since I get a SizeLimitError.REQUEST_SIZE_LIMIT_EXCEEDED when I put more than one operation in the batch service.

It sounds to me that there is an issue in the BatchJobService, or you've used it in a wrong way.
As written in this guide, an AdWords account can have up to 1 GB of uploaded operations across all of its batch jobs that have not yet completed.
So, you shouldn't be faced with the size limit very easily unless your operations are really big.

But do you have a estimated roadmap for when will be this supported?

I can't give you a detailed schedule for that at this moment, but BatchJobUtils may not be available in the experimental branch of this lib very soon, says within 1 month.

I see that for implement the Batch Job upload we will need CurlUtils class or similar rewritten for experimental. Should it be included? It would be nice have a new release of experimental with this features soon, since my client is waiting only for this feature and he doesn't want develop with the old master api

We're not planning to use CurlUtils in this new client lib.
Probably, we'll use Guzzle as we does with ReportDownloader.

[vtsao]

Renaming this bug, adding BJS support to the beta version of this client library is on our timeline for this Q. We'll update this issue as we go along.

[shakaran]

@vtsao do you have some progress with BJS support?

[vtsao]

No updates yet, we're planning to work on it this quarter.

[shakaran]

@vtsao Thanks for the update ;) At least some small fixes were done ;) Keep the good work!

[shakaran]

Hi @vtsao @fiboknacky my client is requesting if we have some progress about this issue, he needs to use this part of code for make a demo and we are trying to avoid the stable version since it is less suitable for new coding. Do you think that we can see progress for this issue soon?

[vtsao]

Hi shakaran@ our current ETA is to have this done and released in the beta lib by the end of this quarter.

[shakaran]

Hi @vtsao just checking the status for this issue since end of Q3 is approaching, there are progress over this issue?

[vtsao]

Yup, still on track to be released by end of this Q.

[vtsao]

Hi @shakaran we've just updated the experimental branch to contain BJS support. Please see:
https://github.com/googleads/googleads-php-lib/blob/experimental/CHANGELOG.md#140-beta

Thanks.

[shakaran]

Thanks @vtsao I will try the new version and I will report if I found some bug or problem. Thanks a lot for your work!