Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(c/driver/postgresql): Implement LIST/LARGE_LIST Writer #1962

Merged
merged 7 commits into from
Jul 4, 2024

Conversation

WillAyd
Copy link
Contributor

@WillAyd WillAyd commented Jul 2, 2024

closes #1882

@@ -105,9 +105,7 @@ class PostgresCopyFieldWriter {
class PostgresCopyFieldTupleWriter : public PostgresCopyFieldWriter {
public:
void AppendChild(std::unique_ptr<PostgresCopyFieldWriter> child) {
int64_t child_i = static_cast<int64_t>(children_.size());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current design makes it so that the array_view is limited to being supplied to the writers through this AppendChild function; however, to support arbitrarily nested writers I think it makes more sense to be moved outside of this

The flip side is now there are a lot of writer.Init() calls placed in the factory function. I also tried to just make the ArrayView part of the constructor for each writer, but that became kind of strange when trying to create the root writer itself.

Probably a better way to structure/refactor this, but decided to live with the code duplication for now to focus on the rest of implementation

ArrowSchemaViewInit(&child_schema_view, schema->children[0], error));
switch (child_schema_view.type) {
case NANOARROW_TYPE_INT32: {
// TODO: likely need to make type_resolver available here
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hinted at in current comment, but I think we need a way to forward the resolver or connection to this part of the code to get this to work

Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable to me. I agree we'll need to pass through the type_resolver somehow

@WillAyd WillAyd marked this pull request as ready for review July 3, 2024 16:15
@github-actions github-actions bot added this to the ADBC Libraries 14 milestone Jul 3, 2024
@WillAyd WillAyd changed the title feat(c/driver/postgresql): Basic Support for Writing LIST types feat(c/driver/postgresql): Implement LIST/LARGE_LIST Writer Jul 3, 2024
Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an organization question

@@ -663,15 +766,15 @@ class PostgresCopyStreamWriter {
return NANOARROW_OK;
}

ArrowErrorCode InitFieldWriters(ArrowError* error) {
ArrowErrorCode InitFieldWriters(const PostgresConnection* conn, ArrowError* error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit but should we pass in the whole connection or maybe just const Resolver&?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might also simplify the amount of setup you have to do in the unit test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea

@lidavidm lidavidm merged commit cf19d33 into apache:main Jul 4, 2024
61 of 63 checks passed
@WillAyd WillAyd deleted the list-writer branch July 4, 2024 03:37
lidavidm pushed a commit that referenced this pull request Jul 10, 2024
…on (#1998)

This is a follow up to #1962,
which required each writer construction to subsequently call `->Init()`
to assign to a protected array_view_ member.

This takes inspiration from the following cpp guideline to have the
class manage its construction more appropriately:


https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#discussion-use-a-factory-function-if-you-need-virtual-behavior-during-initialization
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat(c/driver/postgresql): Add support for bulk ingestion of list types
2 participants