Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datasources #2959

Closed
wants to merge 62 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
97b9718
clean up code
rolnico Mar 14, 2024
7855540
replaced deprecated ZipFile constructor with builder
rolnico Mar 14, 2024
bdd28ca
fix issues
rolnico Mar 14, 2024
87f69d4
fix issue
rolnico Mar 14, 2024
e9381c5
fix issue
rolnico Mar 14, 2024
de680ed
tests modifications
rolnico Mar 15, 2024
045584e
new directory/archive datasource + file information
rolnico Mar 28, 2024
305b853
fix issues
rolnico Mar 28, 2024
47f7315
Merge branch 'main' into nro/datasource
rolnico Mar 28, 2024
af71021
checkstyle
rolnico Mar 28, 2024
62d897e
comment unseless test
rolnico Mar 28, 2024
e25deea
NewDataSource becomes the new DataSource
rolnico Mar 29, 2024
c44c976
fix ArchiveFormat + add ArchiveFormatTest
rolnico Mar 29, 2024
bd4830a
remove unused method
rolnico Mar 29, 2024
b67247d
add error on builder + tests
rolnico Mar 29, 2024
e72ddbb
test archive with subfolder
rolnico Mar 29, 2024
a867c72
coverage
rolnico Mar 29, 2024
53ac48c
coverage
rolnico Mar 29, 2024
e6b7faf
fix test
rolnico Mar 29, 2024
e47aba4
Merge branch 'main' into nro/datasource
rolnico Mar 29, 2024
38fca28
Importer finding issue
geofjamg Jul 25, 2022
f8ad520
update test
rolnico Apr 22, 2024
78c3e64
add parameter in exist/newInputStream methods in DataSources to check…
rolnico Apr 23, 2024
592db3b
Merge branch 'refs/heads/main' into nro/datasource
rolnico Apr 23, 2024
1f8857d
Merge branch 'refs/heads/main' into nro/datasource
rolnico Apr 25, 2024
3dce95a
fixes for review
rolnico May 27, 2024
da258ca
merge main
zamarrenolm May 30, 2024
f9c2d28
Merge branch 'main' into nro/datasource
zamarrenolm May 30, 2024
0e1f76d
reorganize createDataSource methods
rolnico May 31, 2024
92a9cef
Merge branch 'refs/heads/main' into nro/datasource
rolnico May 31, 2024
69073fb
add check if directory is a directory
rolnico May 31, 2024
3611b27
fix issue
rolnico May 31, 2024
d94652f
DataSourceUtil method reorganisation + sourceFormat renamed sourceFor…
rolnico Jun 17, 2024
1e8bee1
fix javadoc
rolnico Jun 17, 2024
e5b6a92
Merge branch 'refs/heads/main' into nro/datasource
rolnico Jun 17, 2024
9e860ae
add getBaseName as deprecated
rolnico Jun 17, 2024
711afe0
fix ZipDataSource coherence
rolnico Jun 17, 2024
0ba39a8
add testConsistencyWithDataSource
rolnico Jun 17, 2024
08e2ef8
fix ExportTest
rolnico Jun 17, 2024
1f8a1b5
fix javadoc
rolnico Jun 17, 2024
6feddf9
rework methods in DataSourceUtil + tests
rolnico Jun 17, 2024
29ace80
fix issues
rolnico Jun 17, 2024
a1877b0
rework methods in DataSourceUtil
rolnico Jun 17, 2024
b9150a0
Merge branch 'refs/heads/main' into nro/datasource
rolnico Jun 18, 2024
6fb50b3
fix findExtension
rolnico Jun 18, 2024
f3a6599
remove double extension management (.iidm.xml, .iidm.json, .iidm.bin)
rolnico Jun 20, 2024
388647d
use new DirectoryDataSource instead of DatasourceUtil.createDirectory…
rolnico Jun 20, 2024
84bd505
revert guessBaseName
rolnico Jun 20, 2024
aa5a803
rename AbstractDataSource + fix calls to getBaseName
rolnico Jun 20, 2024
84d16cb
fix imports
rolnico Jun 20, 2024
398fa29
Merge branch 'refs/heads/main' into nro/datasource
rolnico Jun 20, 2024
1067484
rename datasource classes to be more consistent and precise
rolnico Jun 20, 2024
c7d2615
fix test
rolnico Jun 20, 2024
fb35d89
fix sonar issue
rolnico Jun 20, 2024
f67f5f9
separate exists method in exists+existsStrict to remove checkConsiste…
rolnico Jun 21, 2024
03f9d79
Merge branch 'refs/heads/main' into nro/datasource
rolnico Jun 21, 2024
34763e3
rework some methods in DataSourceUtil + added constructors in Zip/Tar…
rolnico Jun 24, 2024
202c073
fix
rolnico Jun 24, 2024
80be608
fix imports
rolnico Jun 24, 2024
f5734b6
add a test
rolnico Jun 24, 2024
4193d36
add check in CgmesOnDataSource.exists to see if the datasource main f…
rolnico Jun 27, 2024
5939700
Merge branch 'refs/heads/main' into nro/datasource
rolnico Jun 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
test archive with subfolder
Signed-off-by: Nicolas Rol <nicolas.rol@rte-france.com>
rolnico committed Mar 29, 2024
commit e72ddbb3ae3df597fc3a72209b4895804397298f
Original file line number Diff line number Diff line change
@@ -7,17 +7,19 @@
*/
package com.powsybl.commons.datasource;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;

import java.io.IOException;
import java.nio.file.Path;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.Stream;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.*;

/**
* @author Nicolas Rol {@literal <nicolas.rol at rte-france.com>}
@@ -26,6 +28,7 @@ abstract class AbstractArchiveDataSourceTest extends AbstractNewDataSourceTest {
protected final Set<String> filesInArchive = Set.of(
"foo", "foo.txt", "foo.iidm", "foo.xiidm", "foo.v3.iidm", "foo.v3", "foo_bar.iidm", "foo_bar", "bar.iidm", "bar");
protected Set<String> unlistedFiles;
protected String archiveWithSubfolders;

static Stream<Arguments> provideArgumentsForClassAndListingTest() {
return null;
@@ -60,4 +63,24 @@ void testClassAndListing(String baseName, String sourceFormat, ArchiveFormat arc
assertEquals(listedFiles, dataSource.listNames(".*"));
assertEquals(listedBarFiles, dataSource.listNames(".*bar.*"));
}

@Test
void testFileInSubfolder() throws IOException {
// File
Path file = Path.of(Objects.requireNonNull(getClass().getClassLoader().getResource(archiveWithSubfolders)).getPath());

// Create the datasource
DataSource dataSource = DataSourceUtil.createDataSource(
file.getParent(),
file.getFileName().toString(),
"foo");

// Assertions
Set<String> filesInArchive = dataSource.listNames(".*");
assertEquals(2, filesInArchive.size());
assertTrue(filesInArchive.contains("foo.iidm"));
assertTrue(filesInArchive.contains("foo_bar.iidm"));
assertFalse(filesInArchive.contains("foo_baz.iidm"));
assertFalse(filesInArchive.contains("subfolder/foo_baz.iidm"));
}
}
Original file line number Diff line number Diff line change
@@ -54,6 +54,7 @@ void setUp() throws Exception {
fileSystem = Jimfs.newFileSystem(Configuration.unix());
testDir = fileSystem.getPath("/tmp");
Files.createDirectories(testDir);
archiveWithSubfolders = "foo.iidm.tar.gz";
}

@AfterEach
Original file line number Diff line number Diff line change
@@ -46,6 +46,7 @@ void setUp() throws Exception {
fileSystem = Jimfs.newFileSystem(Configuration.unix());
testDir = fileSystem.getPath("/tmp");
Files.createDirectories(testDir);
archiveWithSubfolders = "foo.iidm.zip";
}

@AfterEach
Binary file added commons/src/test/resources/foo.iidm.tar.gz
Binary file not shown.
Binary file added commons/src/test/resources/foo.iidm.zip
Binary file not shown.