Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(csharp/src/Drivers/Apache/Spark): Perform scalar data type conversion for Spark over HTTP #3

Closed
wants to merge 60 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
7e189d1
Add support for username and password authentication
birschick-bq Jul 15, 2024
95a3fd9
Merge branch 'apache:main' into dev/birschick-bq/user-password-auth
birschick-bq Jul 15, 2024
e297113
work-in-progress - support non-Arrow results version 3.3.1
birschick-bq Jul 16, 2024
e4c5f30
make protocolVersion a parameter
birschick-bq Jul 16, 2024
586edcc
updates for HiveServer2Reader and test
birschick-bq Jul 17, 2024
308a78f
remove commented code
birschick-bq Jul 17, 2024
f83c7a6
small refactor/rename
birschick-bq Jul 17, 2024
e320463
Merge remote-tracking branch 'origin/main' into dev/birschick-bq/spar…
birschick-bq Jul 17, 2024
68ee762
refactor more GetRowSet
birschick-bq Jul 17, 2024
df54095
case-insensitive compare
birschick-bq Jul 17, 2024
137caf7
Merge branch 'apache:main' into dev/birschick-bq/user-password-auth
birschick-bq Jul 19, 2024
3e716b9
improvements from code review comments
birschick-bq Jul 19, 2024
c55abaf
improvements from code review comments
birschick-bq Jul 19, 2024
b23b6de
Merge branch 'dev/birschick-bq/user-password-auth' into dev/birschick…
birschick-bq Jul 22, 2024
d359c78
Fix merge error
birschick-bq Jul 22, 2024
92490c7
Fixed compile and test issues
birschick-bq Jul 22, 2024
ba0a966
Merge branch 'dev/birschick-bq/user-password-auth' into dev/birschick…
birschick-bq Jul 22, 2024
69131fa
Many more tests are passing
birschick-bq Jul 24, 2024
4af848c
Detect and handle null values in array columns.
birschick-bq Jul 25, 2024
8e35f95
Detect and handle null values in array columns.
birschick-bq Jul 25, 2024
ed56eb1
Add license.
birschick-bq Jul 25, 2024
828fd95
Improve clarity of tests with protocol/server versions.
birschick-bq Jul 26, 2024
c02241a
Fix test incompatibility.
birschick-bq Jul 26, 2024
bbe81a3
Add negative connection test for authentication and host.
birschick-bq Jul 26, 2024
33799d7
Merge branch 'dev/birschick-bq/user-password-auth' into dev/birschick…
birschick-bq Jul 26, 2024
344cf2a
Update documentation.
birschick-bq Jul 26, 2024
67c14e2
Refactor
birschick-bq Aug 12, 2024
42365d7
Refactor
birschick-bq Aug 12, 2024
c57ed56
correct project change
birschick-bq Aug 12, 2024
442d1c6
self code review #1
birschick-bq Aug 12, 2024
90c0538
self code review #2
birschick-bq Aug 12, 2024
6873bfa
self code review #3
birschick-bq Aug 12, 2024
942c07d
workaround Unsafe.As bug in .net 4.7.2
birschick-bq Aug 12, 2024
c42e5c0
fix alignment
birschick-bq Aug 12, 2024
068d5aa
fix alignment
birschick-bq Aug 12, 2024
be368d1
remove redundant
birschick-bq Aug 12, 2024
46e4c8d
formatting updates
birschick-bq Aug 12, 2024
e549e4a
remove unnecessary using
birschick-bq Aug 12, 2024
8c51e9c
add variant specific validation
birschick-bq Aug 14, 2024
199e558
invert compiler coditional
birschick-bq Aug 14, 2024
0253b64
update documentation
birschick-bq Aug 15, 2024
0b3b6ab
update for code review comments
birschick-bq Aug 30, 2024
9c6f3ed
update for code review comments #2
birschick-bq Aug 30, 2024
3c5a4db
update for code review comments #3
birschick-bq Sep 3, 2024
8102d4b
update for code review comments #4 - data type conversion option
birschick-bq Sep 4, 2024
b7658eb
feat(csharp/src/Drivers/Apache/Spark) Perform scalar data type conver…
birschick-bq Sep 9, 2024
66f4074
add license text
birschick-bq Sep 9, 2024
5c42e26
remove trailing space
birschick-bq Sep 9, 2024
98a4968
some refactoring
birschick-bq Sep 9, 2024
44e8982
improve Impala example
birschick-bq Sep 9, 2024
5b76cfc
Merge remote-tracking branch 'origin/main' into dev/birschick-bq/data…
birschick-bq Sep 10, 2024
43d59ee
Ensure data type conversion is a list of possible conversions.
birschick-bq Sep 10, 2024
6fb9cf2
Add test for data type conversion option parsing.
birschick-bq Sep 10, 2024
569d3b1
Merge remote-tracking branch 'origin/main' into dev/birschick-bq/data…
birschick-bq Sep 11, 2024
8bab697
Corrected tests so that it handles Apache Spark 3.5
birschick-bq Sep 13, 2024
a74cfe4
Merge remote-tracking branch 'origin/main' into dev/birschick-bq/data…
birschick-bq Sep 13, 2024
0ba07c6
Changed enumeration to use flags
birschick-bq Sep 19, 2024
2708aeb
Merge remote-tracking branch 'origin/main' into dev/birschick-bq/data…
birschick-bq Sep 19, 2024
32d7d1f
code review updates, DecimalUtility with exponent support.
birschick-bq Sep 24, 2024
ccc5d3e
code review updates - spelling mistakes corrected.
birschick-bq Sep 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions csharp/src/Drivers/Apache/Hive2/DataTypeConversion.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

using System;

namespace Apache.Arrow.Adbc.Drivers.Apache.Hive2
{
[Flags]
internal enum DataTypeConversion
{
Empty = 0,
None = 1,
Scalar = 2,
}

internal static class DataTypeConversionParser
{
internal const string SupportedList = DataTypeConversionOptions.None + ", " + DataTypeConversionOptions.Scalar;

internal static DataTypeConversion Parse(string? dataTypeConversion)
{
DataTypeConversion result = DataTypeConversion.Empty;

if (string.IsNullOrWhiteSpace(dataTypeConversion))
{
// Default
return DataTypeConversion.Scalar;
}

string[] conversions = dataTypeConversion!.Split(',');
foreach (string? conversion in conversions)
{
result |= (conversion?.Trim().ToLowerInvariant()) switch
{
null or "" => DataTypeConversion.Empty,
DataTypeConversionOptions.None => DataTypeConversion.None,
DataTypeConversionOptions.Scalar => DataTypeConversion.Scalar,
_ => throw new ArgumentOutOfRangeException(nameof(dataTypeConversion), conversion, "Invalid or unsupported data type conversion"),
};
}

if (result.HasFlag(DataTypeConversion.None) && result.HasFlag(DataTypeConversion.Scalar))
{
throw new ArgumentOutOfRangeException(nameof(dataTypeConversion), dataTypeConversion, "Conflicting data type conversion options");
}
// Default
if (result == DataTypeConversion.Empty) result = DataTypeConversion.Scalar;

return result;
}
}
}
Loading
Loading