Adding Doctrine examples

[programarivm]

1. Extraload\Extractor\Doctrine\QueryExtractor is added
2. examples/05_default_doctrine_statement_console.php is created
3. README update

[programarivm]

• Just added a Doctrine PreparedQueryExtractor

[umpirsky]

Great job, I would recommand two things:

• Do one row at a time.
• Merge this two implementations into one.

Thanks!

[umpirsky]

Why numbers? :)

[programarivm]

Okay no problem :)

[umpirsky]

You can share $connectionParams between your examples, it will be easier to configure only once and be able to run all examples that way.

[programarivm]

Yeah, let's do this. Thanks.

[umpirsky]

You should fetch inside extract(), as late as possible.

[programarivm]

Thank you very much for your feedback.

[umpirsky]

For large data sets, and that's common in ETL, fetching all at once can be impossible to process and lead to out of memory issues, especially if doing heavy transformations. It's better if we can extract row by row, and only keep one row in memory at a time.

[umpirsky]

Same applies here.

[umpirsky]

One of this two classes is redundant. I'd refactor it to only QueryExtractor.

(Connection $conn, string $sql, array $values) will do the same job if $values = [].

[programarivm]

I believe my original suggestion brings a bit of awareness to the API user about the fact that they are using either queries or prepared queries.

The logic being run under the hood is a bit different. QueryExtractor runs $conn->query($sql); whereas PreparedQueryExtractor runs $conn->prepare($sql);.

[programarivm]

Thanks for the review! QueryExtractor and PreparedQueryExtractor are merged now into AbstractExtractor if this makes more sense.

[programarivm]

Just added one more Doctrine example using a MapFieldsTransformer -- btw I won't push any more examples to this PR until the MapFieldsTransformer example is reviewed.

[umpirsky]

Why can't we have one class doing both query with and without parameters?

[umpirsky]

I like this transformer, but we can make it better by using https://symfony.com/doc/current/components/property_access.html and make it work with multi-dimensional arrays and objects. What do you think?

Also, it would be great to extract transformet in a separate PR.

[programarivm]

• The MapFieldsTransformer is temporarily removed from this PR
• QueryExtractor and PreparedQueryExtractor are merged into QueryExtractor -- which is now a prepared query with 3 params with the third one being optional

[umpirsky]

My only concern is that $this->data can become huge. But I guess we can optimize later.

Thanks!

[programarivm]

Yeah, it makes sense. $this->data can always be bigger and bigger. Perhaps that is a problem that may be addressed by splitting huge queries into smaller chunks to multiprocess them in parallel.