Skip to content

Commit

Permalink
Merge branch 'master' into 339-metamorphVerifyNoMoreInteractionsPart2
Browse files Browse the repository at this point in the history
  • Loading branch information
blackwinter committed Nov 8, 2021
2 parents eda7269 + 73bedb1 commit def3c60
Show file tree
Hide file tree
Showing 284 changed files with 4,346 additions and 446 deletions.
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,63 +4,63 @@

Metafacture is a toolkit for processing semi-structured data with a focus on library metadata. It provides a versatile set of tools for reading, writing and transforming data. Metafacture can be used as a stand-alone application or as a Java library in other applications. The name Metafacture is a portmanteau of the words *meta*data and manu*facture*.

Metafacture includes a [large number of modules](https://github.com/metafacture/metafacture-documentation/blob/master/flux-commands.md) for operating on semi-structured data. These modules can be combined to build pipelines to perform complex metadata processing tasks. The pipelines can be constructed either in Java code or with the domain-specific language **Flux**. One of the core features of Metafacture is the **Metamorph** module. Metamorph is an xml-based language for specifying transformations of semi-structured data. It can be seamlessly integrated into Java code.
Metafacture includes a [large number of modules](https://github.com/metafacture/metafacture-documentation/blob/master/flux-commands.md) for operating on semi-structured data. These modules can be combined to build pipelines to perform complex metadata processing tasks. The pipelines can be constructed either in Java code or with the domain-specific language **Flux**. One of the core features of Metafacture is the **Metamorph** module. Metamorph is an XML-based language for specifying transformations of semi-structured data. It can be seamlessly integrated into Java code.

At its heart Metafacture is a framework for implementing modules for metadata processing. This makes Metafacture easily extendable with additional modules. The [plugins and tools page](https://github.com/metafacture/metafacture-core/wiki/Plugins-and-Tools) on the wiki shows supplementary packages and projects which extend Metafacture.

Originally, Metafacture was developed as part of the [Culturegraph](http://culturegraph.org) platform but it is developed independently now and used by others, too: [see who uses Metafacture](https://github.com/metafacture/metafacture-core/wiki/Who-uses-Metafacture).
Originally, Metafacture was developed as part of the [Culturegraph](http://www.culturegraph.org) platform but it is developed independently now and used by others, too: [see who uses Metafacture](https://github.com/metafacture/metafacture-core/wiki/Who-uses-Metafacture).

# Getting started

You can either use Metafacture as a stand-alone application or include it as a Java library in your own projects.

## Metafacture as a stand-alone application

If you are only interested in running Flux scripts without doing any Java programming this is the way to go. The instructions assume that you are using a *nix-like shell.
If you are only interested in running Flux scripts without doing any Java programming this is the way to go. The instructions assume that you are using a \*nix-like shell. [See more information in the wiki page about Flux](https://github.com/metafacture/metafacture-core/wiki/Flux-user-guide).

1. Download the latest distribution package from the [metafacture-core/releases](https://github.com/metafacture/metafacture-core/releases) page. Make sure that you do download a distribution package and _not_ a source code package (the file name should include *-dist*).
1. Download the latest distribution package from the [release page](https://github.com/metafacture/metafacture-core/releases). Make sure that you do download a distribution package and *not* a source code package (the file name should include `*-dist*`).

2. Extract the downloaded archive:
```bash
$ tar xzf metafacture-core-VERSION-dist.tar.gz
$ tar xzf metafacture-core-$VERSION-dist.tar.gz
```
This will create a new directory containing a ready-to-use metafacture distribution.
This will create a new directory containing a ready-to-use Metafacture distribution.
3. Change into the newly created directory:
```bash
$ cd metafacture-core-VERSION
$ cd metafacture-core-$VERSION
```
4. Run one of the example scripts:
```bash
$ ./flux.sh examples/read/marc21/read-marc21.flux
```
This example will print a number of marc21 records on standard out.
This example will print a number of MARC 21 records on standard output.

The _examples_ folder contains many more examples which provide a good starting point for learning metafacture. If you have any questions please join our [mailing list](http://lists.dnb.de/mailman/listinfo/metafacture) or use our issue-based discussion forum over at [metafacture-documentation](https://github.com/metafacture/metafacture-documentation).
The `examples` folder contains many more examples which provide a good starting point for learning Metafacture. If you have any questions please join our [mailing list](http://lists.dnb.de/mailman/listinfo/metafacture) or use our issue-based discussion forum over at [metafacture-documentation](https://github.com/metafacture/metafacture-documentation).


## Using Metafacture as a Java libary
## Using Metafacture as a Java library

If you want use Metafacture in your own Java projects all you need to add some dependencies to your project. As of Metafacture 5 the single metafacture-core package has been replaced with a number of domain-specific packages. You can find the list of packages on [Maven Central](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.metafacture%22).
If you want to use Metafacture in your own Java projects all you need is to add some dependencies to your project. As of Metafacture 5, the single metafacture-core package has been replaced with a number of domain-specific packages. You can find the list of packages on [Maven Central](https://search.maven.org/search?q=g:org.metafacture).

Alternatively, you can simply guess the package names from the top-level folders in the source code repository -- they are the same. For instance, if you want to use Metamorph in your project, simply add the following dependency to your `pom.xml`:

```xml
<dependency>
<groupId>org.metafacture</groupId>
<artifactId>metamorph</artifactId>
<version>VERSION</version>
<version>$VERSION</version>
</dependency>
```

or if Gradle is your build tool of choice use:

```groovy
dependencies {
implementation 'org.metafacture:metamorph:VERSION'
implementation 'org.metafacture:metamorph:$VERSION'
}
```

Our integration server automatically publishes successful builds of all branches as snapshot versions on [Sonatype OSS Repository](https://oss.sonatype.org/index.html#nexus-search;quick~metafacture). The version number is derived from the branch name. Snapshot builds from the master branch always have the version "master-SNAPSHOT".
Occasionally, we publish snapshot builds on [Sonatype OSS Repository](https://oss.sonatype.org/index.html#nexus-search;gav~org.metafacture~~~~). The version number is derived from the branch name. Snapshot builds from the master branch always have the version `master-SNAPSHOT`.

<!--
TODO: Link to getting started tutorial
Expand All @@ -77,7 +77,7 @@ $ git clone https://github.com/metafacture/metafacture-core.git
$ cd metafacture-core
```

2. Invoke the gradle-wrapper to download Gradle and build metafacture-core (on Windows call `gradlew.bat install`):
2. Invoke the Gradle wrapper to download Gradle and build metafacture-core (on Windows call `gradlew.bat install` instead):

```bash
$ ./gradlew install
Expand Down
22 changes: 22 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,20 @@ subprojects {
apply plugin: 'checkstyle'

check.dependsOn(editorconfigCheck)
check.dependsOn(javadoc)

sourceCompatibility = 1.8
targetCompatibility = 1.8

tasks.withType(JavaCompile) {
'all -processing -rawtypes -serial'.split().each {
options.compilerArgs << "-Xlint:${it}"
}

options.compilerArgs << '-Werror'
options.encoding = 'UTF-8'
}

task sourceJar(type: Jar) {
dependsOn tasks.classes
from sourceSets.main.allJava
Expand All @@ -81,6 +88,12 @@ subprojects {
group 'build'
}

javadoc {
options {
addBooleanOption 'Xwerror', true
}
}

artifacts {
archives sourceJar
archives javadocJar
Expand All @@ -91,6 +104,15 @@ subprojects {
checkstyleTest.enabled = false
}

test {
systemProperties['org.slf4j.simpleLogger.defaultLogLevel'] = 'warn'

testLogging {
showStandardStreams = true
exceptionFormat = 'full'
}
}

signing {
required {
scmInfo.isRelease() && gradle.taskGraph.hasTask(tasks.uploadArchives)
Expand Down
4 changes: 4 additions & 0 deletions config/checkstyle/checkstyle.xml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
<module name="CustomImportOrder">
<property name="customImportOrderRules" value="SAME_PACKAGE(2)###THIRD_PARTY_PACKAGE###STANDARD_JAVA_PACKAGE###SPECIAL_IMPORTS###STATIC"/>
<property name="separateLineBetweenGroups" value="true"/>
<property name="sortImportsInGroupAlphabetically" value="true"/>
</module>
<module name="CyclomaticComplexity"/>
<module name="DeclarationOrder"/>
Expand Down Expand Up @@ -68,6 +69,7 @@
<module name="InterfaceIsType"/>
<module name="InterfaceTypeParameterName"/>
<module name="JavaNCSS"/>
<module name="JavadocMethod"/>
<module name="JavadocType"/>
<module name="LambdaParameterName"/>
<module name="LeftCurly"/>
Expand All @@ -81,6 +83,7 @@
<module name="MethodTypeParameterName"/>
<module name="MissingCtor"/>
<module name="MissingDeprecated"/>
<module name="MissingJavadocMethod"/>
<module name="MissingOverride"/>
<module name="MissingSwitchDefault"/>
<module name="ModifiedControlVariable"/>
Expand All @@ -97,6 +100,7 @@
<module name="NoClone"/>
<module name="NoEnumTrailingComma"/>
<module name="NoFinalizer"/>
<module name="NonEmptyAtclauseDescription"/>
<module name="NoWhitespaceAfter"/>
<module name="NoWhitespaceBefore"/>
<module name="OneStatementPerLine">
Expand Down
7 changes: 0 additions & 7 deletions metafacture-biblio/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,3 @@ dependencies {
testImplementation 'junit:junit:4.12'
testImplementation 'org.mockito:mockito-core:2.5.5'
}

test {
testLogging {
showStandardStreams = true
exceptionFormat = 'full'
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ public final class AlephMabXmlHandler extends DefaultXmlPipe<StreamReceiver> {
private String currentTag = "";
private StringBuilder builder = new StringBuilder();

/**
* Creates instance of {@link AlephMabXmlHandler}.
*/
public AlephMabXmlHandler() {
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ public final class AseqDecoder extends DefaultObjectPipe<String, StreamReceiver>
private static final int RECORD_IDENTIFIER_BEGIN = 0;
private static final int RECORD_IDENTIFIER_END = 9;

/**
* Creates an instance of {@link AseqDecoder}.
*/
public AseqDecoder() {
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ public class ComarcXmlHandler extends DefaultXmlPipe<StreamReceiver> {
private String currentTag = "";
private StringBuilder builder = new StringBuilder();

/**
* Creates an instance of {@link ComarcXmlHandler}.
*/
public ComarcXmlHandler() {
}

Expand All @@ -86,6 +89,7 @@ else if (RECORD.equals(localName) && NAMESPACE.equals(uri)) {
}

@Override
@SuppressWarnings("fallthrough")
public void endElement(final String uri, final String localName,
final String qName) throws SAXException {
if (SUBFIELD.equals(localName)) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ public final class MabDecoder extends DefaultObjectPipe<String, StreamReceiver>
private static final String ID_TAG = "001 ";
private static final int TAG_LENGTH = 4;

/**
* Creates an instance of {@link MabDecoder}.
*/
public MabDecoder() {
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,42 @@
*/
public interface FieldHandler {

/**
* Reference a field.
*
* @param tag the tag
* @param implDefinedPart the impl defined part
* @param value the value
*/
void referenceField(char[] tag, char[] implDefinedPart, String value);

/**
* Starts a data field.
*
* @param tag the tag
* @param implDefinedPart the impl defined part
* @param indicators the indicators
*/
void startDataField(char[] tag, char[] implDefinedPart, char[] indicators);

/**
* Ends the data field.
*/
void endDataField();

/**
* Sets the impl defined part.
*
* @param implDefinedPart he impl defined part
*/
void additionalImplDefinedPart(char[] implDefinedPart);

/**
* Sets the identifier to a value.
*
* @param identifier the identifier
* @param value the value
*/
void data(char[] identifier, String value);

}
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
package org.metafacture.biblio.iso2709;

/**
* Provides read access to the record label of a ISO 2709:2008 formatted
* Provides read access to the record label of an ISO 2709:2008 formatted
* record. The record label consists of the first 24 octets of the record.
* <p>
* Use {@link LabelBuilder} if write access to the label is required.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,22 +91,47 @@ private int findRecordIdFieldStart() {
return RECORD_ID_MISSING;
}

/**
* Gets the record format of the Label.
*
* @return the record format of the Label
*/
public RecordFormat getRecordFormat() {
return label.getRecordFormat();
}

/**
* Gets the record status of the Label.
*
* @return the record status of the Label
*/
public char getRecordStatus() {
return label.getRecordStatus();
}

/**
* Gets the impl codes.
*
* @return the impl codes
*/
public char[] getImplCodes() {
return label.getImplCodes();
}

/**
* Gets the systems chars of the Label.
*
* @return the system chars
*/
public char[] getSystemChars() {
return label.getSystemChars();
}

/**
* Gets the reserved char of the Label.
*
* @return the reserved char
*/
public char getReservedChar() {
return label.getReservedChar();
}
Expand Down
Loading

0 comments on commit def3c60

Please sign in to comment.