null,
- * in which case only the given document metadata is used for type
- * detection.
- * - * If the document stream supports the - * {@link InputStream#markSupported() mark feature}, then the stream is - * marked and reset to the original position before this method returns. - * Only a limited number of bytes are read from the stream. - *
- * The given document stream is not closed by this method. - *
- * Unlike in the {@link #parse(InputStream, Metadata)} method, the
- * given document metadata is not modified by this method.
+ * Detects the media type of the given document. The type detection is based on the content of
+ * the given document stream and any given document metadata. The document stream can be
+ * null, in which case only the given document metadata is used for type detection.
+ *
+ *
If the document stream supports the {@link InputStream#markSupported() mark feature}, then
+ * the stream is marked and reset to the original position before this method returns. Only a
+ * limited number of bytes are read from the stream.
*
- * @param stream the document stream, or null
+ *
The given document stream is not closed by this method. + * + *
Unlike in the {@link #parse(InputStream, Metadata)} method, the given document metadata is
+ * not modified by this method.
+ *
+ * @param stream the document stream, or null
* @param metadata document metadata
* @return detected media type
* @throws IOException if the stream can not be read
@@ -162,19 +147,17 @@ public String detect(InputStream stream, Metadata metadata) throws IOException {
}
/**
- * Detects the media type of the given document. The type detection is
- * based on the content of the given document stream and the name of the
- * document.
- *
- * If the document stream supports the - * {@link InputStream#markSupported() mark feature}, then the stream is - * marked and reset to the original position before this method returns. - * Only a limited number of bytes are read from the stream. - *
- * The given document stream is not closed by this method. + * Detects the media type of the given document. The type detection is based on the content of + * the given document stream and the name of the document. + * + *
If the document stream supports the {@link InputStream#markSupported() mark feature}, then + * the stream is marked and reset to the original position before this method returns. Only a + * limited number of bytes are read from the stream. + * + *
The given document stream is not closed by this method. * * @param stream the document stream - * @param name document name + * @param name document name * @return detected media type * @throws IOException if the stream can not be read * @since Apache Tika 0.9 @@ -186,15 +169,14 @@ public String detect(InputStream stream, String name) throws IOException { } /** - * Detects the media type of the given document. The type detection is - * based on the content of the given document stream. - *
- * If the document stream supports the - * {@link InputStream#markSupported() mark feature}, then the stream is - * marked and reset to the original position before this method returns. - * Only a limited number of bytes are read from the stream. - *
- * The given document stream is not closed by this method. + * Detects the media type of the given document. The type detection is based on the content of + * the given document stream. + * + *
If the document stream supports the {@link InputStream#markSupported() mark feature}, then + * the stream is marked and reset to the original position before this method returns. Only a + * limited number of bytes are read from the stream. + * + *
The given document stream is not closed by this method. * * @param stream the document stream * @return detected media type @@ -205,16 +187,15 @@ public String detect(InputStream stream) throws IOException { } /** - * Detects the media type of the given document. The type detection is - * based on the first few bytes of a document and the document name. - *
- * For best results at least a few kilobytes of the document data - * are needed. See also the other detect() methods for better - * alternatives when you have more than just the document prefix - * available for type detection. + * Detects the media type of the given document. The type detection is based on the first few + * bytes of a document and the document name. + * + *
For best results at least a few kilobytes of the document data are needed. See also the + * other detect() methods for better alternatives when you have more than just the document + * prefix available for type detection. * * @param prefix first few bytes of the document - * @param name document name + * @param name document name * @return detected media type * @since Apache Tika 0.9 */ @@ -229,13 +210,12 @@ public String detect(byte[] prefix, String name) { } /** - * Detects the media type of the given document. The type detection is - * based on the first few bytes of a document. - *
- * For best results at least a few kilobytes of the document data - * are needed. See also the other detect() methods for better - * alternatives when you have more than just the document prefix - * available for type detection. + * Detects the media type of the given document. The type detection is based on the first few + * bytes of a document. + * + *
For best results at least a few kilobytes of the document data are needed. See also the + * other detect() methods for better alternatives when you have more than just the document + * prefix available for type detection. * * @param prefix first few bytes of the document * @return detected media type @@ -252,12 +232,11 @@ public String detect(byte[] prefix) { } /** - * Detects the media type of the file at the given path. The type - * detection is based on the document content and a potential known - * file extension. - *
- * Use the {@link #detect(String)} method when you want to detect the - * type of the document without actually accessing the file. + * Detects the media type of the file at the given path. The type detection is based on the + * document content and a potential known file extension. + * + *
Use the {@link #detect(String)} method when you want to detect the type of the document + * without actually accessing the file. * * @param path the path of the file * @return detected media type @@ -271,11 +250,11 @@ public String detect(Path path) throws IOException { } /** - * Detects the media type of the given file. The type detection is - * based on the document content and a potential known file extension. - *
- * Use the {@link #detect(String)} method when you want to detect the - * type of the document without actually accessing the file. + * Detects the media type of the given file. The type detection is based on the document content + * and a potential known file extension. + * + *
Use the {@link #detect(String)} method when you want to detect the type of the document + * without actually accessing the file. * * @param file the file * @return detected media type @@ -284,19 +263,18 @@ public String detect(Path path) throws IOException { */ public String detect(File file) throws IOException { Metadata metadata = new Metadata(); - try (@SuppressWarnings("deprecation") InputStream stream = TikaInputStream - .get(file, metadata)) { + try (@SuppressWarnings("deprecation") + InputStream stream = TikaInputStream.get(file, metadata)) { return detect(stream, metadata); } } /** - * Detects the media type of the resource at the given URL. The type - * detection is based on the document content and a potential known - * file extension included in the URL. - *
- * Use the {@link #detect(String)} method when you want to detect the - * type of the document without actually accessing the URL. + * Detects the media type of the resource at the given URL. The type detection is based on the + * document content and a potential known file extension included in the URL. + * + *
Use the {@link #detect(String)} method when you want to detect the type of the document + * without actually accessing the URL. * * @param url the URL of the resource * @return detected media type @@ -310,11 +288,11 @@ public String detect(URL url) throws IOException { } /** - * Detects the media type of a document with the given file name. - * The type detection is based on known file name extensions. - *
- * The given name can also be a URL or a full file path. In such cases - * only the file name part of the string is used for type detection. + * Detects the media type of a document with the given file name. The type detection is based on + * known file name extensions. + * + *
The given name can also be a URL or a full file path. In such cases only the file name + * part of the string is used for type detection. * * @param name the file name of the document * @return detected media type @@ -330,11 +308,11 @@ public String detect(String name) { /** * Translate the given text String to and from the given languages. * - * @param text The text to translate. + * @param text The text to translate. * @param sourceLanguage The input text language (for example, "hi"). * @param targetLanguage The desired output language (for example, "fr"). - * @return The translated text. If translation is unavailable (client keys not set), returns - * the same text back. + * @return The translated text. If translation is unavailable (client keys not set), returns the + * same text back. * @see org.apache.tika.language.translate.Translator */ public String translate(String text, String sourceLanguage, String targetLanguage) { @@ -346,13 +324,13 @@ public String translate(String text, String sourceLanguage, String targetLanguag } /** - * Translate the given text String to the given language, attempting to auto-detect the - * source language. + * Translate the given text String to the given language, attempting to auto-detect the source + * language. * - * @param text The text to translate. + * @param text The text to translate. * @param targetLanguage The desired output language (for example, "en"). - * @return The translated text. If translation is unavailable (client keys not set), returns - * the same text back. + * @return The translated text. If translation is unavailable (client keys not set), returns the + * same text back. * @see org.apache.tika.language.translate.Translator */ public String translate(String text, String targetLanguage) { @@ -363,18 +341,16 @@ public String translate(String text, String targetLanguage) { } } - /** - * Parses the given document and returns the extracted text content. - * Input metadata like a file name or a content type hint can be passed - * in the given metadata instance. Metadata information extracted from - * the document is returned in that same metadata instance. - *
- * The returned reader will be responsible for closing the given stream. - * The stream and any associated resources will be closed at or before - * the time when the {@link Reader#close()} method is called. - * - * @param stream the document to be parsed + * Parses the given document and returns the extracted text content. Input metadata like a file + * name or a content type hint can be passed in the given metadata instance. Metadata + * information extracted from the document is returned in that same metadata instance. + * + *
The returned reader will be responsible for closing the given stream. The stream and any + * associated resources will be closed at or before the time when the {@link Reader#close()} + * method is called. + * + * @param stream the document to be parsed * @param metadata where document's metadata will be populated * @return extracted text content * @throws IOException if the document can not be read or parsed @@ -387,10 +363,10 @@ public Reader parse(InputStream stream, Metadata metadata) throws IOException { /** * Parses the given document and returns the extracted text content. - *
- * The returned reader will be responsible for closing the given stream. - * The stream and any associated resources will be closed at or before - * the time when the {@link Reader#close()} method is called. + * + *
The returned reader will be responsible for closing the given stream. The stream and any + * associated resources will be closed at or before the time when the {@link Reader#close()} + * method is called. * * @param stream the document to be parsed * @return extracted text content @@ -402,11 +378,11 @@ public Reader parse(InputStream stream) throws IOException { /** * Parses the file at the given path and returns the extracted text content. - *
- * Metadata information extracted from the document is returned in - * the supplied metadata instance. * - * @param path the path of the file to be parsed + *
Metadata information extracted from the document is returned in the supplied metadata + * instance. + * + * @param path the path of the file to be parsed * @param metadata where document's metadata will be populated * @return extracted text content * @throws IOException if the file can not be read or parsed @@ -429,18 +405,19 @@ public Reader parse(Path path) throws IOException { /** * Parses the given file and returns the extracted text content. - *
- * Metadata information extracted from the document is returned in - * the supplied metadata instance. * - * @param file the file to be parsed + *
Metadata information extracted from the document is returned in the supplied metadata + * instance. + * + * @param file the file to be parsed * @param metadata where document's metadata will be populated * @return extracted text content * @throws IOException if the file can not be read or parsed * @see #parse(Path) */ public Reader parse(File file, Metadata metadata) throws IOException { - @SuppressWarnings("deprecation") InputStream stream = TikaInputStream.get(file, metadata); + @SuppressWarnings("deprecation") + InputStream stream = TikaInputStream.get(file, metadata); return parse(stream, metadata); } @@ -457,8 +434,7 @@ public Reader parse(File file) throws IOException { } /** - * Parses the resource at the given URL and returns the extracted - * text content. + * Parses the resource at the given URL and returns the extracted text content. * * @param url the URL of the resource to be parsed * @return extracted text content @@ -471,23 +447,21 @@ public Reader parse(URL url) throws IOException { } /** - * Parses the given document and returns the extracted text content. - * The given input stream is closed by this method. - *
- * To avoid unpredictable excess memory use, the returned string contains - * only up to {@link #getMaxStringLength()} first characters extracted - * from the input document. Use the {@link #setMaxStringLength(int)} - * method to adjust this limitation. - *
- * NOTE: Unlike most other Tika methods that take an - * {@link InputStream}, this method will close the given stream for - * you as a convenience. With other methods you are still responsible - * for closing the stream or a wrapper instance returned by Tika. - * - * @param stream the document to be parsed + * Parses the given document and returns the extracted text content. The given input stream is + * closed by this method. + * + *
To avoid unpredictable excess memory use, the returned string contains only up to {@link + * #getMaxStringLength()} first characters extracted from the input document. Use the {@link + * #setMaxStringLength(int)} method to adjust this limitation. + * + *
NOTE: Unlike most other Tika methods that take an {@link InputStream}, + * this method will close the given stream for you as a convenience. With other methods you are + * still responsible for closing the stream or a wrapper instance returned by Tika. + * + * @param stream the document to be parsed * @param metadata document metadata * @return extracted text content - * @throws IOException if the document can not be read + * @throws IOException if the document can not be read * @throws TikaException if the document can not be parsed */ public String parseToString(InputStream stream, Metadata metadata) @@ -496,24 +470,21 @@ public String parseToString(InputStream stream, Metadata metadata) } /** - * Parses the given document and returns the extracted text content. - * The given input stream is closed by this method. This method lets - * you control the maxStringLength per call. - *
- * To avoid unpredictable excess memory use, the returned string contains - * only up to maxLength (parameter) first characters extracted - * from the input document. - *
- * NOTE: Unlike most other Tika methods that take an - * {@link InputStream}, this method will close the given stream for - * you as a convenience. With other methods you are still responsible - * for closing the stream or a wrapper instance returned by Tika. - * - * @param stream the document to be parsed - * @param metadata document metadata + * Parses the given document and returns the extracted text content. The given input stream is + * closed by this method. This method lets you control the maxStringLength per call. + * + *
To avoid unpredictable excess memory use, the returned string contains only up to + * maxLength (parameter) first characters extracted from the input document. + * + *
NOTE: Unlike most other Tika methods that take an {@link InputStream}, + * this method will close the given stream for you as a convenience. With other methods you are + * still responsible for closing the stream or a wrapper instance returned by Tika. + * + * @param stream the document to be parsed + * @param metadata document metadata * @param maxLength maximum length of the returned string * @return extracted text content - * @throws IOException if the document can not be read + * @throws IOException if the document can not be read * @throws TikaException if the document can not be parsed */ public String parseToString(InputStream stream, Metadata metadata, int maxLength) @@ -535,22 +506,20 @@ public String parseToString(InputStream stream, Metadata metadata, int maxLength } /** - * Parses the given document and returns the extracted text content. - * The given input stream is closed by this method. - *
- * To avoid unpredictable excess memory use, the returned string contains - * only up to {@link #getMaxStringLength()} first characters extracted - * from the input document. Use the {@link #setMaxStringLength(int)} - * method to adjust this limitation. - *
- * NOTE: Unlike most other Tika methods that take an - * {@link InputStream}, this method will close the given stream for - * you as a convenience. With other methods you are still responsible - * for closing the stream or a wrapper instance returned by Tika. + * Parses the given document and returns the extracted text content. The given input stream is + * closed by this method. + * + *
To avoid unpredictable excess memory use, the returned string contains only up to {@link + * #getMaxStringLength()} first characters extracted from the input document. Use the {@link + * #setMaxStringLength(int)} method to adjust this limitation. + * + *
NOTE: Unlike most other Tika methods that take an {@link InputStream}, + * this method will close the given stream for you as a convenience. With other methods you are + * still responsible for closing the stream or a wrapper instance returned by Tika. * * @param stream the document to be parsed * @return extracted text content - * @throws IOException if the document can not be read + * @throws IOException if the document can not be read * @throws TikaException if the document can not be parsed */ public String parseToString(InputStream stream) throws IOException, TikaException { @@ -559,15 +528,14 @@ public String parseToString(InputStream stream) throws IOException, TikaExceptio /** * Parses the file at the given path and returns the extracted text content. - *
- * To avoid unpredictable excess memory use, the returned string contains - * only up to {@link #getMaxStringLength()} first characters extracted - * from the input document. Use the {@link #setMaxStringLength(int)} - * method to adjust this limitation. + * + *
To avoid unpredictable excess memory use, the returned string contains only up to {@link + * #getMaxStringLength()} first characters extracted from the input document. Use the {@link + * #setMaxStringLength(int)} method to adjust this limitation. * * @param path the path of the file to be parsed * @return extracted text content - * @throws IOException if the file can not be read + * @throws IOException if the file can not be read * @throws TikaException if the file can not be parsed */ public String parseToString(Path path) throws IOException, TikaException { @@ -578,36 +546,34 @@ public String parseToString(Path path) throws IOException, TikaException { /** * Parses the given file and returns the extracted text content. - *
- * To avoid unpredictable excess memory use, the returned string contains - * only up to {@link #getMaxStringLength()} first characters extracted - * from the input document. Use the {@link #setMaxStringLength(int)} - * method to adjust this limitation. + * + *
To avoid unpredictable excess memory use, the returned string contains only up to {@link + * #getMaxStringLength()} first characters extracted from the input document. Use the {@link + * #setMaxStringLength(int)} method to adjust this limitation. * * @param file the file to be parsed * @return extracted text content - * @throws IOException if the file can not be read + * @throws IOException if the file can not be read * @throws TikaException if the file can not be parsed * @see #parseToString(Path) */ public String parseToString(File file) throws IOException, TikaException { Metadata metadata = new Metadata(); - @SuppressWarnings("deprecation") InputStream stream = TikaInputStream.get(file, metadata); + @SuppressWarnings("deprecation") + InputStream stream = TikaInputStream.get(file, metadata); return parseToString(stream, metadata); } /** - * Parses the resource at the given URL and returns the extracted - * text content. - *
- * To avoid unpredictable excess memory use, the returned string contains - * only up to {@link #getMaxStringLength()} first characters extracted - * from the input document. Use the {@link #setMaxStringLength(int)} - * method to adjust this limitation. + * Parses the resource at the given URL and returns the extracted text content. + * + *
To avoid unpredictable excess memory use, the returned string contains only up to {@link
+ * #getMaxStringLength()} first characters extracted from the input document. Use the {@link
+ * #setMaxStringLength(int)} method to adjust this limitation.
*
* @param url the URL of the resource to be parsed
* @return extracted text content
- * @throws IOException if the resource can not be read
+ * @throws IOException if the resource can not be read
* @throws TikaException if the resource can not be parsed
*/
public String parseToString(URL url) throws IOException, TikaException {
@@ -617,8 +583,7 @@ public String parseToString(URL url) throws IOException, TikaException {
}
/**
- * Returns the maximum length of strings returned by the
- * parseToString methods.
+ * Returns the maximum length of strings returned by the parseToString methods.
*
* @return maximum string length, or -1 if the limit has been disabled
* @since Apache Tika 0.7
@@ -628,11 +593,9 @@ public int getMaxStringLength() {
}
/**
- * Sets the maximum length of strings returned by the parseToString
- * methods.
+ * Sets the maximum length of strings returned by the parseToString methods.
*
- * @param maxStringLength maximum string length,
- * or -1 to disable this limit
+ * @param maxStringLength maximum string length, or -1 to disable this limit
* @since Apache Tika 0.7
*/
public void setMaxStringLength(int maxStringLength) {
@@ -669,7 +632,7 @@ public Translator getTranslator() {
return translator;
}
- //--------------------------------------------------------------< Object >
+ // --------------------------------------------------------------< Object >
public String toString() {
return getString();
@@ -678,8 +641,9 @@ public String toString() {
public static String getString() {
String version = null;
- try (InputStream stream = Tika.class
- .getResourceAsStream("/META-INF/maven/org.apache.tika/tika-core/pom.properties")) {
+ try (InputStream stream =
+ Tika.class.getResourceAsStream(
+ "/META-INF/maven/org.apache.tika/tika-core/pom.properties")) {
if (stream != null) {
Properties properties = new Properties();
properties.load(stream);
@@ -694,5 +658,4 @@ public static String getString() {
return "Apache Tika";
}
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/concurrent/ConfigurableThreadPoolExecutor.java b/tika-core/src/main/java/org/apache/tika/concurrent/ConfigurableThreadPoolExecutor.java
index 1f7c4a0567..f3a5ffadf0 100644
--- a/tika-core/src/main/java/org/apache/tika/concurrent/ConfigurableThreadPoolExecutor.java
+++ b/tika-core/src/main/java/org/apache/tika/concurrent/ConfigurableThreadPoolExecutor.java
@@ -24,9 +24,8 @@
* @since Apache Tika 1.11
*/
public interface ConfigurableThreadPoolExecutor extends ExecutorService {
-
+
public void setMaximumPoolSize(int threads);
-
- public void setCorePoolSize(int threads);
+ public void setCorePoolSize(int threads);
}
diff --git a/tika-core/src/main/java/org/apache/tika/concurrent/SimpleThreadPoolExecutor.java b/tika-core/src/main/java/org/apache/tika/concurrent/SimpleThreadPoolExecutor.java
index a4385e279e..6a7ee32cf4 100644
--- a/tika-core/src/main/java/org/apache/tika/concurrent/SimpleThreadPoolExecutor.java
+++ b/tika-core/src/main/java/org/apache/tika/concurrent/SimpleThreadPoolExecutor.java
@@ -25,11 +25,16 @@
*
* @since Apache Tika 1.11
*/
-public class SimpleThreadPoolExecutor extends ThreadPoolExecutor implements
- ConfigurableThreadPoolExecutor {
+public class SimpleThreadPoolExecutor extends ThreadPoolExecutor
+ implements ConfigurableThreadPoolExecutor {
public SimpleThreadPoolExecutor() {
- super(1, 2, 0L, TimeUnit.SECONDS, new LinkedBlockingQueue<>(),
+ super(
+ 1,
+ 2,
+ 0L,
+ TimeUnit.SECONDS,
+ new LinkedBlockingQueue<>(),
r -> new Thread(r, "Tika Executor Thread"));
}
}
diff --git a/tika-core/src/main/java/org/apache/tika/config/ConfigBase.java b/tika-core/src/main/java/org/apache/tika/config/ConfigBase.java
index 405294faed..448f8820e9 100644
--- a/tika-core/src/main/java/org/apache/tika/config/ConfigBase.java
+++ b/tika-core/src/main/java/org/apache/tika/config/ConfigBase.java
@@ -29,23 +29,21 @@
import java.util.Locale;
import java.util.Map;
import java.util.Set;
-
+import org.apache.tika.exception.TikaConfigException;
+import org.apache.tika.exception.TikaException;
+import org.apache.tika.utils.XMLReaderUtils;
import org.w3c.dom.Element;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;
-import org.apache.tika.exception.TikaConfigException;
-import org.apache.tika.exception.TikaException;
-import org.apache.tika.utils.XMLReaderUtils;
-
-
public abstract class ConfigBase {
private static Class[] SUPPORTED_PRIMITIVES =
- new Class[]{String.class, boolean.class, long.class, int.class, double.class,
- float.class};
+ new Class[] {
+ String.class, boolean.class, long.class, int.class, double.class, float.class
+ };
/**
* Use this to build a single class, where the user specifies the instance class, e.g.
@@ -81,8 +79,9 @@ protected static compositeClass,
- String itemName, Class compositeClass,
+ String itemName,
+ Class c
} catch (TikaException e) {
throw new TikaConfigException("problem loading xml to dom", e);
}
- return buildComposite(compositeElementName, compositeClass, itemName, itemClass,
- properties);
+ return buildComposite(
+ compositeElementName, compositeClass, itemName, itemClass, properties);
}
- protected static compositeClass,
- String itemName, Class compositeClass,
+ String itemName,
+ Class c
P composite = (P) constructor.newInstance(components);
setParams(composite, child, new HashSet<>(), itemName);
return composite;
- } catch (NoSuchMethodException | InvocationTargetException |
- InstantiationException | IllegalAccessException e) {
+ } catch (NoSuchMethodException
+ | InvocationTargetException
+ | InstantiationException
+ | IllegalAccessException e) {
throw new TikaConfigException("can't build composite class", e);
}
}
@@ -168,8 +175,8 @@ protected static c
throw new TikaConfigException("could not find " + compositeElementName);
}
- private static
- * TikaConfig will call initialize on Initializable classes after
- * setting the parameters for non-statically service loaded classes.
- *
- * TikaConfig will call checkInitialization on all Initializables,
- * whether loaded statically
+ * Components that must do special processing across multiple fields at initialization time should
+ * implement this interface.
+ *
+ * TikaConfig will call initialize on Initializable classes after setting the parameters for
+ * non-statically service loaded classes.
+ *
+ * TikaConfig will call checkInitialization on all Initializables, whether loaded statically
*/
public interface Initializable {
@@ -38,15 +36,10 @@ public interface Initializable {
*/
void initialize(Map
- * Note that this class does not use the {@link LoadErrorHandler}, a
- * {@link ClassNotFoundException} is always returned for unknown
- * classes or classes of the wrong type
+ * Loads and returns the named service class that's expected to implement the given interface.
+ *
+ * Note that this class does not use the {@link LoadErrorHandler}, a {@link
+ * ClassNotFoundException} is always returned for unknown classes or classes of the wrong type
*
* @param iface service interface
- * @param name service class name
+ * @param name service class name
* @return service class
- * @throws ClassNotFoundException if the service class can not be found
- * or does not implement the given interface
+ * @throws ClassNotFoundException if the service class can not be found or does not implement
+ * the given interface
* @see Class#forName(String, boolean, ClassLoader)
* @since Apache Tika 1.1
*/
@@ -225,10 +228,8 @@ public As of versions after 2.4.1, this removes duplicate classes
*
* @param iface service provider interface
* @return available service providers
@@ -256,7 +257,7 @@ public
- * Note that you should not access this class directly.
- * Instead the OSGi environment (if present) will automatically invoke the
- * methods of this class based on the Bundle-Activator setting in the bundle
- * manifest.
+ * Bundle activator that adjust the class loading mechanism of the {@link ServiceLoader} class to
+ * work correctly in an OSGi environment.
+ *
+ * Note that you should not access this class directly. Instead the OSGi
+ * environment (if present) will automatically invoke the methods of this class based on the
+ * Bundle-Activator setting in the bundle manifest.
*
* @since Apache Tika 0.9
*/
@@ -44,7 +42,8 @@ public class TikaActivator implements BundleActivator, ServiceTrackerCustomizer
private ServiceTracker parserTracker;
private BundleContext bundleContext;
- //-----------------------------------------------------< BundleActivator >
+
+ // -----------------------------------------------------< BundleActivator >
public void start(final BundleContext context) throws Exception {
bundleContext = context;
@@ -73,12 +72,10 @@ public Object addingService(ServiceReference reference) {
return service;
}
- public void modifiedService(ServiceReference reference, Object service) {
- }
+ public void modifiedService(ServiceReference reference, Object service) {}
public void removedService(ServiceReference reference, Object service) {
ServiceLoader.removeService(reference);
bundleContext.ungetService(reference);
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/config/TikaConfig.java b/tika-core/src/main/java/org/apache/tika/config/TikaConfig.java
index e68ad10d65..7fe261430c 100644
--- a/tika-core/src/main/java/org/apache/tika/config/TikaConfig.java
+++ b/tika-core/src/main/java/org/apache/tika/config/TikaConfig.java
@@ -39,15 +39,6 @@
import java.util.concurrent.ExecutorService;
import java.util.concurrent.atomic.AtomicInteger;
import javax.imageio.spi.ServiceRegistry;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.w3c.dom.Document;
-import org.w3c.dom.Element;
-import org.w3c.dom.Node;
-import org.w3c.dom.NodeList;
-import org.xml.sax.SAXException;
-
import org.apache.tika.concurrent.ConfigurableThreadPoolExecutor;
import org.apache.tika.concurrent.SimpleThreadPoolExecutor;
import org.apache.tika.detect.CompositeDetector;
@@ -81,16 +72,21 @@
import org.apache.tika.utils.AnnotationUtils;
import org.apache.tika.utils.StringUtils;
import org.apache.tika.utils.XMLReaderUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Document;
+import org.w3c.dom.Element;
+import org.w3c.dom.Node;
+import org.w3c.dom.NodeList;
+import org.xml.sax.SAXException;
-/**
- * Parse xml config file.
- */
+/** Parse xml config file. */
public class TikaConfig {
- public static int DEFAULT_MAX_JSON_STRING_FIELD_LENGTH = 20_000_000;//jackson's default
+ public static int DEFAULT_MAX_JSON_STRING_FIELD_LENGTH = 20_000_000; // jackson's default
public static String MAX_JSON_STRING_FIELD_LENGTH_ELEMENT_NAME = "maxJsonStringFieldLength";
- //use this to look for unneeded instantiations of TikaConfig
+ // use this to look for unneeded instantiations of TikaConfig
protected static final AtomicInteger TIMES_INSTANTIATED = new AtomicInteger();
private static final Logger LOG = LoggerFactory.getLogger(TikaConfig.class);
@@ -124,13 +120,16 @@ public TikaConfig(Path path, ServiceLoader loader)
public TikaConfig(File file) throws TikaException, IOException, SAXException {
this(XMLReaderUtils.buildDOM(file.toPath()));
}
+
public TikaConfig(File file, ServiceLoader loader)
throws TikaException, IOException, SAXException {
this(XMLReaderUtils.buildDOM(file.toPath()), loader);
}
+
public TikaConfig(URL url) throws TikaException, IOException, SAXException {
this(url, ServiceLoader.getContextClassLoader());
}
+
public TikaConfig(URL url, ClassLoader loader) throws TikaException, IOException, SAXException {
this(XMLReaderUtils.buildDOM(url.toString()).getDocumentElement(), loader);
}
@@ -184,15 +183,14 @@ private TikaConfig(Element element, ServiceLoader loader) throws TikaException,
}
/**
- * Creates a Tika configuration from the built-in media type rules
- * and all the {@link Parser} implementations available through the
- * {@link ServiceRegistry service provider mechanism} in the given
- * class loader.
+ * Creates a Tika configuration from the built-in media type rules and all the {@link Parser}
+ * implementations available through the {@link ServiceRegistry service provider mechanism} in
+ * the given class loader.
*
- * @param loader the class loader through which parser implementations
- * are loaded, or If one of these have a value, try to resolve it relative to file
- * system or classpath. If XML config is not specified, initialize from the built-in media
- * type rules and all the {@link Parser} implementations available through
- * the {@link ServiceRegistry service provider mechanism} in the context
- * class loader of the current thread. If one of these have a value, try to resolve it relative to file system or classpath.
+ *
+ * If XML config is not specified, initialize from the built-in media type rules and all the
+ * {@link Parser} implementations available through the {@link ServiceRegistry service provider
+ * mechanism} in the context class loader of the current thread.
+ *
+ * @throws IOException if the configuration can not be read
* @throws TikaException if problem with MimeTypes or parsing XML config
*/
public TikaConfig() throws TikaException, IOException {
@@ -281,17 +280,16 @@ public TikaConfig() throws TikaException, IOException {
this.autoDetectParserConfig = AutoDetectParserConfig.load(element);
setMaxJsonStringFieldLength(element);
} catch (SAXException e) {
- throw new TikaException("Specified Tika configuration has syntax errors: " + config,
- e);
+ throw new TikaException(
+ "Specified Tika configuration has syntax errors: " + config, e);
}
}
TIMES_INSTANTIATED.incrementAndGet();
}
/**
- *
* @return maximum field length when serializing String fields in Tika's metadata or metadata
- * list into JSON
+ * list into JSON
*/
public static int getMaxJsonStringFieldLength() {
return MAX_JSON_STRING_FIELD_LENGTH;
@@ -305,8 +303,9 @@ private void setMaxJsonStringFieldLength(Element properties) throws TikaConfigEx
try {
MAX_JSON_STRING_FIELD_LENGTH = Integer.parseInt(n.getTextContent());
} catch (NumberFormatException e) {
- throw new TikaConfigException(MAX_JSON_STRING_FIELD_LENGTH_ELEMENT_NAME + " " +
- "is not an integer", e);
+ throw new TikaConfigException(
+ MAX_JSON_STRING_FIELD_LENGTH_ELEMENT_NAME + " " + "is not an integer",
+ e);
}
return;
}
@@ -328,8 +327,12 @@ protected static CompositeEncodingDetector getDefaultEncodingDetector(ServiceLoa
protected static CompositeRenderer getDefaultRenderer(ServiceLoader loader) {
return new CompositeRenderer(loader);
}
- private static CompositeParser getDefaultParser(MimeTypes types, ServiceLoader loader,
- EncodingDetector encodingDetector, Renderer renderer) {
+
+ private static CompositeParser getDefaultParser(
+ MimeTypes types,
+ ServiceLoader loader,
+ EncodingDetector encodingDetector,
+ Renderer renderer) {
return new DefaultParser(types.getMediaTypeRegistry(), loader, encodingDetector, renderer);
}
@@ -379,9 +382,9 @@ private static String getText(Node node) {
}
/**
- * Provides a default configuration (TikaConfig). Currently creates a
- * new instance each time it's called; we may be able to have it
- * return a shared instance once it is completely immutable.
+ * Provides a default configuration (TikaConfig). Currently creates a new instance each time
+ * it's called; we may be able to have it return a shared instance once it is completely
+ * immutable.
*
* @return default configuration
*/
@@ -406,9 +409,8 @@ private static Element getChild(Element element, String name) {
return null;
}
- private static List
- * Detectors are loaded and returned in a specified order, of user supplied
- * followed by non-MimeType Tika, followed by the Tika MimeType class.
- * If you need to control the order of the Detectors, you should instead
- * construct your own {@link CompositeDetector} and pass in the list
+ * A composite detector based on all the {@link Detector} implementations available through the
+ * {@link ServiceRegistry service provider mechanism}.
+ *
+ * Detectors are loaded and returned in a specified order, of user supplied followed by
+ * non-MimeType Tika, followed by the Tika MimeType class. If you need to control the order of the
+ * Detectors, you should instead construct your own {@link CompositeDetector} and pass in the list
* of Detectors in the required order.
*
* @since Apache Tika 0.9
*/
public class DefaultDetector extends CompositeDetector {
- /**
- * Serial version UID
- */
+ /** Serial version UID */
private static final long serialVersionUID = -8170114575326908027L;
- private transient final ServiceLoader loader;
- public DefaultDetector(MimeTypes types, ServiceLoader loader,
- Collection
- * If an {@link OverrideDetector} is loaded, it takes precedence over
- * all other detectors.
+ * Finds all statically loadable detectors and sort the list by name, rather than discovery
+ * order. Detectors are used in the given order, so put the Tika parsers last so that non-Tika
+ * (user supplied) parsers can take precedence.
+ *
+ * If an {@link OverrideDetector} is loaded, it takes precedence over all other detectors.
*
* @param loader service loader
* @return ordered list of statically loadable detectors
*/
- private static List
+ * A composite encoding detector based on all the {@link EncodingDetector} implementations available
+ * through the {@link ServiceRegistry service provider mechanism}. Those loaded via the service
+ * provider mechanism are ordered by how they appear in the file, if there is a single service file.
+ * If multiple, there is no guarantee of order.
+ *
*
- * If you need to control the order of the Detectors, you should instead
- * construct your own {@link CompositeDetector} and pass in the list
- * of Detectors in the required order.
+ *
+ * If you need to control the order of the Detectors, you should instead construct your own
+ * {@link CompositeDetector} and pass in the list of Detectors in the required order.
*
* @since Apache Tika 1.15
*/
@@ -45,10 +44,9 @@ public DefaultEncodingDetector(ServiceLoader loader) {
super(loader.loadServiceProviders(EncodingDetector.class));
}
- public DefaultEncodingDetector(ServiceLoader loader,
- Collection
- * If the document input stream is not available, then the first
- * argument may be
- * The given input metadata is only read, not modified, by the detector.
+ * Detects the content type of the given input document. Returns If the document input stream is not available, then the first argument may be The given input metadata is only read, not modified, by the detector.
+ *
+ * @param input document input stream, or
- * If the document input stream is not available, then the first
- * argument may be If the document input stream is not available, then the first argument may be
- * The given input metadata is only read, not modified, by the detector.
*
- * @param input text document input stream, or The given input metadata is only read, not modified, by the detector.
+ *
+ * @param input text document input stream, or
- * If this is run against any other type of InputStream, it will spool
- * up to {@link #maxBytes} to disk and then run the detector.
- *
- * As with all detectors, mark must be supported.
- *
- * If you want to use file's mime type in the parse, e.g.
- * to select the parser in AutoDetectParser, set {@link FileCommandDetector#setUseMime(boolean)}
- * to true. The default behavior is to store the value as {@link FileCommandDetector#FILE_MIME}
- * but rely on other detectors for the "active" mime used by Tika.
+ * This runs the linux 'file' command against a file. If this is called on a TikaInputStream, it
+ * will use the underlying Path or spool the full file to disk and then run file against that.
+ *
+ * If this is run against any other type of InputStream, it will spool up to {@link #maxBytes} to
+ * disk and then run the detector.
+ *
+ * As with all detectors, mark must be supported.
+ *
+ * If you want to use file's mime type in the parse, e.g. to select the parser in
+ * AutoDetectParser, set {@link FileCommandDetector#setUseMime(boolean)} to true. The default
+ * behavior is to store the value as {@link FileCommandDetector#FILE_MIME} but rely on other
+ * detectors for the "active" mime used by Tika.
*/
public class FileCommandDetector implements Detector {
- //TODO: file has some diff mimes names for some very common mimes
- //should we map file mimes to Tika mimes, e.g. text/xml -> application/xml??
+ // TODO: file has some diff mimes names for some very common mimes
+ // should we map file mimes to Tika mimes, e.g. text/xml -> application/xml??
public static Property FILE_MIME = Property.externalText("file:mime");
private static final Logger LOGGER = LoggerFactory.getLogger(FileCommandDetector.class);
@@ -75,14 +72,13 @@ public static boolean checkHasFile() {
return checkHasFile(DEFAULT_FILE_COMMAND_PATH);
}
-
public static boolean checkHasFile(String fileCommandPath) {
- String[] commandline = new String[]{fileCommandPath, "-v"};
+ String[] commandline = new String[] {fileCommandPath, "-v"};
return ExternalParser.check(commandline);
}
/**
- * @param input document input stream, or
- * Because this works on bytes, not characters, by default any string
- * matching is done as ISO_8859_1. To use an explicit different
- * encoding, supply a type other than "string" / "stringignorecase"
+ * Content type detection based on magic bytes, i.e. type-specific patterns near the beginning of
+ * the document input stream.
+ *
+ * Because this works on bytes, not characters, by default any string matching is done as
+ * ISO_8859_1. To use an explicit different encoding, supply a type other than "string" /
+ * "stringignorecase"
*
* @since Apache Tika 0.3
*/
public class MagicDetector implements Detector {
/**
- * The matching media type. Returned by the
- * {@link #detect(InputStream, Metadata)} method if a match is found.
+ * The matching media type. Returned by the {@link #detect(InputStream, Metadata)} method if a
+ * match is found.
*/
private final MediaType type;
- /**
- * Length of the comparison window.
- */
+
+ /** Length of the comparison window. */
private final int length;
+
/**
- * The magic match pattern. If this byte pattern is equal to the
- * possibly bit-masked bytes from the input stream, then the type
- * detection succeeds and the configured {@link #type} is returned.
+ * The magic match pattern. If this byte pattern is equal to the possibly bit-masked bytes from
+ * the input stream, then the type detection succeeds and the configured {@link #type} is
+ * returned.
*/
private final byte[] pattern;
+
/**
- * Length of the pattern, which in the case of regular expressions will
- * not be the same as the comparison window length.
+ * Length of the pattern, which in the case of regular expressions will not be the same as the
+ * comparison window length.
*/
private final int patternLength;
- /**
- * True if pattern is a regular expression, false otherwise.
- */
+
+ /** True if pattern is a regular expression, false otherwise. */
private final boolean isRegex;
- /**
- * True if we're doing a case-insensitive string match, false otherwise.
- */
+
+ /** True if we're doing a case-insensitive string match, false otherwise. */
private final boolean isStringIgnoreCase;
- /**
- * Bit mask that is applied to the source bytes before pattern matching.
- */
+
+ /** Bit mask that is applied to the source bytes before pattern matching. */
private final byte[] mask;
+
/**
- * First offset (inclusive) of the comparison window within the
- * document input stream. Greater than or equal to zero.
+ * First offset (inclusive) of the comparison window within the document input stream. Greater
+ * than or equal to zero.
*/
private final int offsetRangeBegin;
+
/**
- * Last offset (inclusive) of the comparison window within the document
- * input stream. Greater than or equal to the
- * {@link #offsetRangeBegin first offset}.
- *
- * Note that this is not the offset of the last byte read from
- * the document stream. Instead, the last window of bytes to be compared
- * starts at this offset.
+ * Last offset (inclusive) of the comparison window within the document input stream. Greater
+ * than or equal to the {@link #offsetRangeBegin first offset}.
+ *
+ * Note that this is not the offset of the last byte read from the document stream.
+ * Instead, the last window of bytes to be compared starts at this offset.
*/
private final int offsetRangeEnd;
/**
- * Creates a detector for input documents that have the exact given byte
- * pattern at the beginning of the document stream.
+ * Creates a detector for input documents that have the exact given byte pattern at the
+ * beginning of the document stream.
*
- * @param type matching media type
+ * @param type matching media type
* @param pattern magic match pattern
*/
public MagicDetector(MediaType type, byte[] pattern) {
@@ -103,42 +100,46 @@ public MagicDetector(MediaType type, byte[] pattern) {
}
/**
- * Creates a detector for input documents that have the exact given byte
- * pattern at the given offset of the document stream.
+ * Creates a detector for input documents that have the exact given byte pattern at the given
+ * offset of the document stream.
*
- * @param type matching media type
+ * @param type matching media type
* @param pattern magic match pattern
- * @param offset offset of the pattern match
+ * @param offset offset of the pattern match
*/
public MagicDetector(MediaType type, byte[] pattern, int offset) {
this(type, pattern, null, offset, offset);
}
/**
- * Creates a detector for input documents that meet the specified magic
- * match. {@code pattern} must NOT be a regular expression.
- * Constructor maintained for legacy reasons.
+ * Creates a detector for input documents that meet the specified magic match. {@code pattern}
+ * must NOT be a regular expression. Constructor maintained for legacy reasons.
*/
- public MagicDetector(MediaType type, byte[] pattern, byte[] mask, int offsetRangeBegin,
- int offsetRangeEnd) {
+ public MagicDetector(
+ MediaType type, byte[] pattern, byte[] mask, int offsetRangeBegin, int offsetRangeEnd) {
this(type, pattern, mask, false, offsetRangeBegin, offsetRangeEnd);
}
- /**
- * Creates a detector for input documents that meet the specified
- * magic match.
- */
- public MagicDetector(MediaType type, byte[] pattern, byte[] mask, boolean isRegex,
- int offsetRangeBegin, int offsetRangeEnd) {
+ /** Creates a detector for input documents that meet the specified magic match. */
+ public MagicDetector(
+ MediaType type,
+ byte[] pattern,
+ byte[] mask,
+ boolean isRegex,
+ int offsetRangeBegin,
+ int offsetRangeEnd) {
this(type, pattern, mask, isRegex, false, offsetRangeBegin, offsetRangeEnd);
}
- /**
- * Creates a detector for input documents that meet the specified
- * magic match.
- */
- public MagicDetector(MediaType type, byte[] pattern, byte[] mask, boolean isRegex,
- boolean isStringIgnoreCase, int offsetRangeBegin, int offsetRangeEnd) {
+ /** Creates a detector for input documents that meet the specified magic match. */
+ public MagicDetector(
+ MediaType type,
+ byte[] pattern,
+ byte[] mask,
+ boolean isRegex,
+ boolean isStringIgnoreCase,
+ int offsetRangeBegin,
+ int offsetRangeEnd) {
if (type == null) {
throw new IllegalArgumentException("Matching media type is null");
} else if (pattern == null) {
@@ -183,8 +184,8 @@ public MagicDetector(MediaType type, byte[] pattern, byte[] mask, boolean isRege
this.offsetRangeEnd = offsetRangeEnd;
}
- public static MagicDetector parse(MediaType mediaType, String type, String offset, String value,
- String mask) {
+ public static MagicDetector parse(
+ MediaType mediaType, String type, String offset, String value, String mask) {
int start = 0;
int end = 0;
if (offset != null) {
@@ -204,8 +205,14 @@ public static MagicDetector parse(MediaType mediaType, String type, String offse
maskBytes = decodeValue(mask, type);
}
- return new MagicDetector(mediaType, patternBytes, maskBytes, type.equals("regex"),
- type.equals("stringignorecase"), start, end);
+ return new MagicDetector(
+ mediaType,
+ patternBytes,
+ maskBytes,
+ type.equals("regex"),
+ type.equals("stringignorecase"),
+ start,
+ end);
}
private static byte[] decodeValue(String value, String type) {
@@ -241,29 +248,43 @@ private static byte[] decodeValue(String value, String type) {
decoded = tmpVal.getBytes(UTF_8);
break;
case "host16":
- case "little16": {
- int i = Integer.parseInt(tmpVal, radix);
- decoded = new byte[]{(byte) (i & 0x00FF), (byte) (i >> 8)};
- break;
- }
- case "big16": {
- int i = Integer.parseInt(tmpVal, radix);
- decoded = new byte[]{(byte) (i >> 8), (byte) (i & 0x00FF)};
- break;
- }
+ case "little16":
+ {
+ int i = Integer.parseInt(tmpVal, radix);
+ decoded = new byte[] {(byte) (i & 0x00FF), (byte) (i >> 8)};
+ break;
+ }
+ case "big16":
+ {
+ int i = Integer.parseInt(tmpVal, radix);
+ decoded = new byte[] {(byte) (i >> 8), (byte) (i & 0x00FF)};
+ break;
+ }
case "host32":
- case "little32": {
- long i = Long.parseLong(tmpVal, radix);
- decoded = new byte[]{(byte) ((i & 0x000000FF)), (byte) ((i & 0x0000FF00) >> 8),
- (byte) ((i & 0x00FF0000) >> 16), (byte) ((i & 0xFF000000) >> 24)};
- break;
- }
- case "big32": {
- long i = Long.parseLong(tmpVal, radix);
- decoded = new byte[]{(byte) ((i & 0xFF000000) >> 24), (byte) ((i & 0x00FF0000) >> 16),
- (byte) ((i & 0x0000FF00) >> 8), (byte) ((i & 0x000000FF))};
- break;
- }
+ case "little32":
+ {
+ long i = Long.parseLong(tmpVal, radix);
+ decoded =
+ new byte[] {
+ (byte) ((i & 0x000000FF)),
+ (byte) ((i & 0x0000FF00) >> 8),
+ (byte) ((i & 0x00FF0000) >> 16),
+ (byte) ((i & 0xFF000000) >> 24)
+ };
+ break;
+ }
+ case "big32":
+ {
+ long i = Long.parseLong(tmpVal, radix);
+ decoded =
+ new byte[] {
+ (byte) ((i & 0xFF000000) >> 24),
+ (byte) ((i & 0x00FF0000) >> 16),
+ (byte) ((i & 0x0000FF00) >> 8),
+ (byte) ((i & 0x000000FF))
+ };
+ break;
+ }
}
return decoded;
}
@@ -295,8 +316,9 @@ private static byte[] decodeString(String value, String type) {
i++;
} else {
int j = i + 1;
- while ((j < i + 4) && (j < value.length()) &&
- (Character.isDigit(value.charAt(j)))) {
+ while ((j < i + 4)
+ && (j < value.length())
+ && (Character.isDigit(value.charAt(j)))) {
j++;
}
decoded.write(Short.decode("0" + value.substring(i + 1, j)).byteValue());
@@ -333,7 +355,7 @@ private static byte[] decodeString(String value, String type) {
}
/**
- * @param input document input stream, or
- * If a pattern matches the given name, then the media type associated with
- * that pattern is returned as the likely content type of the input document.
- * Otherwise the returned type is
- * See the {@link #detect(InputStream, Metadata)} method for more details
- * of the matching algorithm.
+ * Content type detection based on the resource name. An instance of this class contains a set of
+ * regular expression patterns that are matched against the resource name potentially given as a
+ * part of the input metadata.
+ *
+ * If a pattern matches the given name, then the media type associated with that pattern is
+ * returned as the likely content type of the input document. Otherwise the returned type is See the {@link #detect(InputStream, Metadata)} method for more details of the matching
+ * algorithm.
*
* @since Apache Tika 0.3
*/
public class NameDetector implements Detector {
- /**
- * The regular expression patterns used for type detection.
- */
+ /** The regular expression patterns used for type detection. */
private final Map
- * If a resource name is given, then it is first processed as follows.
+ * Detects the content type of an input document based on the document name given in the input
+ * metadata. The RESOURCE_NAME_KEY attribute of the given input metadata is expected to contain
+ * the name (normally a file name or a URL) of the input document.
+ *
+ * If a resource name is given, then it is first processed as follows.
+ *
*
- * The resulting name string (if any) is then matched in sequence against
- * all the configured name patterns. If a match is found, then the (first)
- * matching media type is returned.
*
- * @param input ignored
+ * The resulting name string (if any) is then matched in sequence against all the configured
+ * name patterns. If a match is found, then the (first) matching media type is returned.
+ *
+ * @param input ignored
* @param metadata input metadata, possibly with a RESOURCE_NAME_KEY value
* @return detected media type, or
- * This is also required to override detection by some parsers
- * via {@link TikaCoreProperties#CONTENT_TYPE_PARSER_OVERRIDE}.
+ * Use this to force a content type detection via the {@link
+ * TikaCoreProperties#CONTENT_TYPE_USER_OVERRIDE} key in the metadata object.
+ *
+ * This is also required to override detection by some parsers via {@link
+ * TikaCoreProperties#CONTENT_TYPE_PARSER_OVERRIDE}.
*
* @deprecated after 2.5.0 this functionality was moved to the CompositeDetector
*/
diff --git a/tika-core/src/main/java/org/apache/tika/detect/TextDetector.java b/tika-core/src/main/java/org/apache/tika/detect/TextDetector.java
index 96583be7c2..8cfad3f564 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/TextDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/TextDetector.java
@@ -19,46 +19,39 @@
import java.io.IOException;
import java.io.InputStream;
import java.util.Arrays;
-
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
/**
- * Content type detection of plain text documents. This detector looks at the
- * beginning of the document input stream and considers the document to be
- * a text document if no ASCII (ISO-Latin-1, UTF-8, etc.) control bytes are
- * found. As a special case some control bytes (up to 2% of all characters)
- * are also allowed in a text document if it also contains no or just a few
- * (less than 10%) characters above the 7-bit ASCII range.
- *
- * Note that text documents with a character encoding like UTF-16 are better
- * detected with {@link MagicDetector} and an appropriate magic byte pattern.
+ * Content type detection of plain text documents. This detector looks at the beginning of the
+ * document input stream and considers the document to be a text document if no ASCII (ISO-Latin-1,
+ * UTF-8, etc.) control bytes are found. As a special case some control bytes (up to 2% of all
+ * characters) are also allowed in a text document if it also contains no or just a few (less than
+ * 10%) characters above the 7-bit ASCII range.
+ *
+ * Note that text documents with a character encoding like UTF-16 are better detected with {@link
+ * MagicDetector} and an appropriate magic byte pattern.
*
* @since Apache Tika 0.3
*/
public class TextDetector implements Detector {
- /**
- * Serial version UID
- */
+ /** Serial version UID */
private static final long serialVersionUID = 4774601079503507765L;
- /**
- * The number of bytes from the beginning of the document stream
- * to test for control bytes.
- */
+ /** The number of bytes from the beginning of the document stream to test for control bytes. */
private static final int DEFAULT_NUMBER_OF_BYTES_TO_TEST = 512;
/**
- * Lookup table for all the ASCII/ISO-Latin/UTF-8/etc. control bytes
- * in the range below 0x20 (the space character). If an entry in this
- * table is
- * The contents of this lookup table are based on the following definition
- * from section 4 of the "Content-Type Processing Model" Internet-draft
- * (true then that byte is very
+ * unlikely to occur in a plain text document.
+ *
+ * The contents of this lookup table are based on the following definition from section 4 of
+ * the "Content-Type Processing Model" Internet-draft (draft-abarth-mime-sniff-01).
+ *
*
- * This definition of control characters is based on section 4 of the
- * "Content-Type Processing Model" Internet-draft
- * (This definition of control characters is based on section 4 of the "Content-Type
+ * Processing Model" Internet-draft (draft-abarth-mime-sniff-01).
+ *
*
- * The name differs from the precedence of {@link Parser#getSupportedTypes(ParseContext)}
- * so that parser implementations may also choose to implement this interface.
+ * Returns the set of media types supported by this embedder when used with the given parse
+ * context.
+ *
+ * The name differs from the precedence of {@link Parser#getSupportedTypes(ParseContext)} so
+ * that parser implementations may also choose to implement this interface.
*
* @param context parse context
* @return immutable set of media types
@@ -48,46 +47,45 @@ public interface Embedder extends Serializable {
Set
- * The given document stream is consumed but not closed by this method. The
- * responsibility to close the stream remains on the caller.
- *
- * Information about the parsing context can be passed in the context
- * parameter. See the parser implementations for the kinds of context
- * information they expect.
- *
- * In general implementations should favor preserving the source file's metadata
- * unless an update to a field is explicitly defined in the Metadata object.
- * More specifically:
+ * Embeds related document metadata from the given metadata object into the given output stream.
+ *
+ * The given document stream is consumed but not closed by this method. The responsibility to
+ * close the stream remains on the caller.
+ *
+ * Information about the parsing context can be passed in the context parameter. See the
+ * parser implementations for the kinds of context information they expect.
+ *
+ * In general implementations should favor preserving the source file's metadata unless an
+ * update to a field is explicitly defined in the Metadata object. More specifically:
+ *
* null to automatically select the context class loader.
+ * The default context class loader to use for all threads, or null to
+ * automatically select the context class loader.
*/
private static volatile ClassLoader CONTEXT_CLASS_LOADER = null;
+
private final ClassLoader loader;
private final LoadErrorHandler handler;
private final InitializableProblemHandler initializableProblemHandler;
private final boolean dynamic;
- public ServiceLoader(ClassLoader loader, LoadErrorHandler handler,
- InitializableProblemHandler initializableProblemHandler, boolean dynamic) {
+ public ServiceLoader(
+ ClassLoader loader,
+ LoadErrorHandler handler,
+ InitializableProblemHandler initializableProblemHandler,
+ boolean dynamic) {
this.loader = loader;
this.handler = handler;
this.initializableProblemHandler = initializableProblemHandler;
this.dynamic = dynamic;
-
}
+
public ServiceLoader(ClassLoader loader, LoadErrorHandler handler, boolean dynamic) {
this(loader, handler, InitializableProblemHandler.WARN, dynamic);
}
@@ -80,24 +84,27 @@ public ServiceLoader(ClassLoader loader, LoadErrorHandler handler) {
}
public ServiceLoader(ClassLoader loader) {
- this(loader,
- Boolean.getBoolean("org.apache.tika.service.error.warn") ? LoadErrorHandler.WARN :
- LoadErrorHandler.IGNORE);
+ this(
+ loader,
+ Boolean.getBoolean("org.apache.tika.service.error.warn")
+ ? LoadErrorHandler.WARN
+ : LoadErrorHandler.IGNORE);
}
public ServiceLoader() {
- this(getContextClassLoader(),
- Boolean.getBoolean("org.apache.tika.service.error.warn") ? LoadErrorHandler.WARN :
- LoadErrorHandler.IGNORE, true);
+ this(
+ getContextClassLoader(),
+ Boolean.getBoolean("org.apache.tika.service.error.warn")
+ ? LoadErrorHandler.WARN
+ : LoadErrorHandler.IGNORE,
+ true);
}
/**
- * Returns the context class loader of the current thread. If such
- * a class loader is not available, then the loader of this class or
- * finally the system class loader is returned.
+ * Returns the context class loader of the current thread. If such a class loader is not
+ * available, then the loader of this class or finally the system class loader is returned.
*
- * @return context class loader, or null if no loader
- * is available
+ * @return context class loader, or null if no loader is available
* @see TIKA-441
*/
static ClassLoader getContextClassLoader() {
@@ -112,12 +119,11 @@ static ClassLoader getContextClassLoader() {
}
/**
- * Sets the context class loader to use for all threads that access
- * this class. Used for example in an OSGi environment to avoid problems
- * with the default context class loader.
+ * Sets the context class loader to use for all threads that access this class. Used for example
+ * in an OSGi environment to avoid problems with the default context class loader.
*
- * @param loader default context class loader,
- * or null to automatically pick the loader
+ * @param loader default context class loader, or null to automatically pick the
+ * loader
*/
public static void setContextClassLoader(ClassLoader loader) {
CONTEXT_CLASS_LOADER = loader;
@@ -166,8 +172,7 @@ public InitializableProblemHandler getInitializableProblemHandler() {
}
/**
- * Returns an input stream for reading the specified resource from the
- * configured class loader.
+ * Returns an input stream for reading the specified resource from the configured class loader.
*
* @param name resource name
* @return input stream, or null if the resource was not found
@@ -192,18 +197,16 @@ public ClassLoader getLoader() {
}
/**
- * Loads and returns the named service class that's expected to implement
- * the given interface.
- * null for no parsers
+ * @param loader the class loader through which parser implementations are loaded, or null
+ * for no parsers
* @throws MimeTypeException if the built-in media type rules are broken
- * @throws IOException if the built-in media type rules can not be read
+ * @throws IOException if the built-in media type rules can not be read
* @since Apache Tika 0.8
*/
public TikaConfig(ClassLoader loader) throws MimeTypeException, IOException {
@@ -210,20 +208,21 @@ public TikaConfig(ClassLoader loader) throws MimeTypeException, IOException {
}
/**
- * Creates a default Tika configuration.
- * First checks whether an XML config file is specified, either in
+ * Creates a default Tika configuration. First checks whether an XML config file is specified,
+ * either in
+ *
*
- *
- * null
+ * @param input text document input stream, or null
* @param metadata input metadata for the document
* @return the detected Charset or null if no charset could be detected
* @throws IOException
@@ -66,9 +62,10 @@ public Charset detect(InputStream input, Metadata metadata) throws IOException {
Charset detected = detector.detect(input, metadata);
if (detected != null) {
metadata.set(TikaCoreProperties.DETECTED_ENCODING, detected.name());
- //if this has been set by a leaf detector, do not overwrite
- if (! detector.getClass().getSimpleName().equals("CompositeEncodingDetector")) {
- metadata.set(TikaCoreProperties.ENCODING_DETECTOR,
+ // if this has been set by a leaf detector, do not overwrite
+ if (!detector.getClass().getSimpleName().equals("CompositeEncodingDetector")) {
+ metadata.set(
+ TikaCoreProperties.ENCODING_DETECTOR,
detector.getClass().getSimpleName());
}
return detected;
@@ -84,8 +81,8 @@ public Listapplication/octet-stream if the type of the document
- * can not be detected.
- * null. Otherwise the detector may
- * read bytes from the start of the stream to help in type detection.
- * The given stream is guaranteed to support the
- * {@link InputStream#markSupported() mark feature} and the detector
- * is expected to {@link InputStream#mark(int) mark} the stream before
- * reading any bytes from it, and to {@link InputStream#reset() reset}
- * the stream before returning. The stream must not be closed by the
- * detector.
- * application/octet-stream
+ * if the type of the document can not be detected.
+ *
+ * null
+ * . Otherwise the detector may read bytes from the start of the stream to help in type
+ * detection. The given stream is guaranteed to support the {@link InputStream#markSupported()
+ * mark feature} and the detector is expected to {@link InputStream#mark(int) mark} the stream
+ * before reading any bytes from it, and to {@link InputStream#reset() reset} the stream before
+ * returning. The stream must not be closed by the detector.
*
- * @param input document input stream, or null
+ * null
* @param metadata input metadata for the document
* @return detected media type, or application/octet-stream
* @throws IOException if the document input stream could not be read
*/
MediaType detect(InputStream input, Metadata metadata) throws IOException;
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/EmptyDetector.java b/tika-core/src/main/java/org/apache/tika/detect/EmptyDetector.java
index 9f996301ce..c76d79e210 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/EmptyDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/EmptyDetector.java
@@ -18,22 +18,16 @@
import java.io.IOException;
import java.io.InputStream;
-
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
-/**
- * Dummy detector that returns application/octet-stream for all documents.
- */
+/** Dummy detector that returns application/octet-stream for all documents. */
public class EmptyDetector implements Detector {
- /**
- * Singleton instance of this class.
- */
+ /** Singleton instance of this class. */
public static final EmptyDetector INSTANCE = new EmptyDetector();
public MediaType detect(InputStream input, Metadata metadata) throws IOException {
return MediaType.OCTET_STREAM;
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/EncodingDetector.java b/tika-core/src/main/java/org/apache/tika/detect/EncodingDetector.java
index 9dbad4c277..be60018448 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/EncodingDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/EncodingDetector.java
@@ -20,39 +20,35 @@
import java.io.InputStream;
import java.io.Serializable;
import java.nio.charset.Charset;
-
import org.apache.tika.metadata.Metadata;
/**
- * Character encoding detector. Implementations of this interface use
- * various heuristics to detect the character encoding of a text document
- * based on given input metadata or the first few bytes of the document stream.
+ * Character encoding detector. Implementations of this interface use various heuristics to detect
+ * the character encoding of a text document based on given input metadata or the first few bytes of
+ * the document stream.
*
* @since Apache Tika 0.4
*/
public interface EncodingDetector extends Serializable {
/**
- * Detects the character encoding of the given text document, or
- * null if the encoding of the document can not be detected.
- * null. Otherwise the detector may
- * read bytes from the start of the stream to help in encoding detection.
- * The given stream is guaranteed to support the
- * {@link InputStream#markSupported() mark feature} and the detector
- * is expected to {@link InputStream#mark(int) mark} the stream before
- * reading any bytes from it, and to {@link InputStream#reset() reset}
- * the stream before returning. The stream must not be closed by the
+ * Detects the character encoding of the given text document, or null if the
+ * encoding of the document can not be detected.
+ *
+ * null
+ * . Otherwise the detector may read bytes from the start of the stream to help in
+ * encoding detection. The given stream is guaranteed to support the {@link
+ * InputStream#markSupported() mark feature} and the detector is expected to {@link
+ * InputStream#mark(int) mark} the stream before reading any bytes from it, and to {@link
+ * InputStream#reset() reset} the stream before returning. The stream must not be closed by the
* detector.
- * null
+ * null
* @param metadata input metadata for the document
* @return detected character encoding, or null
* @throws IOException if the document input stream could not be read
*/
Charset detect(InputStream input, Metadata metadata) throws IOException;
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/FileCommandDetector.java b/tika-core/src/main/java/org/apache/tika/detect/FileCommandDetector.java
index 42349faec1..a3a7211537 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/FileCommandDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/FileCommandDetector.java
@@ -22,10 +22,6 @@
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
import org.apache.tika.config.Field;
import org.apache.tika.io.BoundedInputStream;
import org.apache.tika.io.TemporaryResources;
@@ -38,26 +34,27 @@
import org.apache.tika.utils.FileProcessResult;
import org.apache.tika.utils.ProcessUtils;
import org.apache.tika.utils.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
- * This runs the linux 'file' command against a file. If
- * this is called on a TikaInputStream, it will use the underlying Path
- * or spool the full file to disk and then run file against that.
- * null
+ * @param input document input stream, or null
* @param metadata input metadata for the document
* @return mime as identified by the file command or application/octet-stream otherwise
* @throws IOException
@@ -101,8 +97,8 @@ public MediaType detect(InputStream input, Metadata metadata) throws IOException
}
TikaInputStream tis = TikaInputStream.cast(input);
if (tis != null) {
- //spool the full file to disk, if called with a TikaInputStream
- //and there is no underlying file
+ // spool the full file to disk, if called with a TikaInputStream
+ // and there is no underlying file
return detectOnPath(tis.getPath(), metadata);
}
@@ -119,8 +115,12 @@ public MediaType detect(InputStream input, Metadata metadata) throws IOException
private MediaType detectOnPath(Path path, Metadata metadata) throws IOException {
String[] args =
- new String[]{ProcessUtils.escapeCommandLine(fileCommandPath), "-b", "--mime-type",
- ProcessUtils.escapeCommandLine(path.toAbsolutePath().toString())};
+ new String[] {
+ ProcessUtils.escapeCommandLine(fileCommandPath),
+ "-b",
+ "--mime-type",
+ ProcessUtils.escapeCommandLine(path.toAbsolutePath().toString())
+ };
ProcessBuilder builder = new ProcessBuilder(args);
FileProcessResult result = ProcessUtils.execute(builder, timeoutMs, 10000, 10000);
if (result.isTimeout()) {
@@ -149,8 +149,8 @@ private MediaType detectOnPath(Path path, Metadata metadata) throws IOException
@Field
public void setFilePath(String fileCommandPath) {
- //this opens up a potential command vulnerability.
- //Don't ever let an untrusted user set this.
+ // this opens up a potential command vulnerability.
+ // Don't ever let an untrusted user set this.
this.fileCommandPath = fileCommandPath;
checkHasFile(this.fileCommandPath);
}
@@ -163,10 +163,10 @@ public void setUseMime(boolean useMime) {
public boolean isUseMime() {
return useMime;
}
+
/**
- * If this is not called on a TikaInputStream, this detector
- * will spool up to this many bytes to a file to be detected
- * by the 'file' command.
+ * If this is not called on a TikaInputStream, this detector will spool up to this many bytes to
+ * a file to be detected by the 'file' command.
*
* @param maxBytes
*/
@@ -179,5 +179,4 @@ public void setMaxBytes(int maxBytes) {
public void setTimeoutMs(long timeoutMs) {
this.timeoutMs = timeoutMs;
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/MagicDetector.java b/tika-core/src/main/java/org/apache/tika/detect/MagicDetector.java
index bb9ec1da02..b8d5205e21 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/MagicDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/MagicDetector.java
@@ -27,75 +27,72 @@
import java.util.Locale;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
-
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
/**
- * Content type detection based on magic bytes, i.e. type-specific patterns
- * near the beginning of the document input stream.
- * null
+ * @param input document input stream, or null
* @param metadata ignored
*/
public MediaType detect(InputStream input, Metadata metadata) throws IOException {
@@ -425,13 +447,18 @@ public int getLength() {
}
/**
- * Returns a string representation of the Detection Rule.
- * Should sort nicely by type and details, as we sometimes
- * compare these.
+ * Returns a string representation of the Detection Rule. Should sort nicely by type and
+ * details, as we sometimes compare these.
*/
public String toString() {
// Needs to be unique, as these get compared.
- return "Magic Detection for " + type + " looking for " + pattern.length + " bytes = " +
- this.pattern + " mask = " + this.mask;
+ return "Magic Detection for "
+ + type
+ + " looking for "
+ + pattern.length
+ + " bytes = "
+ + this.pattern
+ + " mask = "
+ + this.mask;
}
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/NNExampleModelDetector.java b/tika-core/src/main/java/org/apache/tika/detect/NNExampleModelDetector.java
index bcbf48feb5..76fe42dad6 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/NNExampleModelDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/NNExampleModelDetector.java
@@ -27,12 +27,10 @@
import java.net.URL;
import java.nio.file.Path;
import java.util.Objects;
-
+import org.apache.tika.mime.MediaType;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.tika.mime.MediaType;
-
public class NNExampleModelDetector extends TrainedModelDetector {
private static final String EXAMPLE_NNMODEL_FILE = "tika-example.nnmodel";
@@ -68,16 +66,13 @@ public void loadDefaultModels(InputStream modelStream) {
// add this model into map of trained models.
super.registerModels(nnBuilder.getType(), nnBuilder.build());
}
-
}
} catch (IOException e) {
throw new RuntimeException("Unable to read the default media type registry", e);
}
}
- /**
- * this method gets overwritten to register load neural network models
- */
+ /** this method gets overwritten to register load neural network models */
@Override
public void loadDefaultModels(ClassLoader classLoader) {
if (classLoader == null) {
@@ -91,22 +86,20 @@ public void loadDefaultModels(ClassLoader classLoader) {
// Get the core URL, and all the extensions URLs
URL modelURL = classLoader.getResource(classPrefix + EXAMPLE_NNMODEL_FILE);
- Objects.requireNonNull(modelURL,
- "required resource " + classPrefix + EXAMPLE_NNMODEL_FILE + " not found");
+ Objects.requireNonNull(
+ modelURL, "required resource " + classPrefix + EXAMPLE_NNMODEL_FILE + " not found");
try (InputStream stream = modelURL.openStream()) {
loadDefaultModels(stream);
} catch (IOException e) {
throw new RuntimeException("Unable to read the default media type registry", e);
}
-
}
/**
- * read the comments where the model configuration is written, e.g the
- * number of inputs, hiddens and output please ensure the first char in the
- * given string is # In this example grb model file, there are 4 elements 1)
- * type 2) number of input units 3) number of hidden units. 4) number of
- * output units.
+ * read the comments where the model configuration is written, e.g the number of inputs, hiddens
+ * and output please ensure the first char in the given string is # In this example grb model
+ * file, there are 4 elements 1) type 2) number of input units 3) number of hidden units. 4)
+ * number of output units.
*/
private void readDescription(final NNTrainedModelBuilder builder, final String line) {
int numInputs;
@@ -130,8 +123,8 @@ private void readDescription(final NNTrainedModelBuilder builder, final String l
}
/**
- * Read the next line for the model parameters and populate the build which
- * later will be used to instantiate the instance of TrainedModel
+ * Read the next line for the model parameters and populate the build which later will be used
+ * to instantiate the instance of TrainedModel
*
* @param builder
* @param line
diff --git a/tika-core/src/main/java/org/apache/tika/detect/NNTrainedModel.java b/tika-core/src/main/java/org/apache/tika/detect/NNTrainedModel.java
index 73ee560db4..c0f06e9846 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/NNTrainedModel.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/NNTrainedModel.java
@@ -25,8 +25,8 @@ public class NNTrainedModel extends TrainedModel {
private final float[][] Theta1;
private final float[][] Theta2;
- public NNTrainedModel(final int nInput, final int nHidden, final int nOutput,
- final float[] nn_params) {
+ public NNTrainedModel(
+ final int nInput, final int nHidden, final int nOutput, final float[] nn_params) {
this.numOfInputs = nInput;
this.numOfHidden = nHidden;
this.numOfOutputs = nOutput;
@@ -64,8 +64,7 @@ public double predict(double[] unseen) {
}
/**
- * The given input vector of unseen is m=(256 + 1) * n= 1 this returns a
- * prediction probability
+ * The given input vector of unseen is m=(256 + 1) * n= 1 this returns a prediction probability
*/
@Override
public float predict(float[] unseen) {
@@ -74,7 +73,7 @@ public float predict(float[] unseen) {
int i, j;
int m = this.Theta1.length;
int n = this.Theta1[0].length;
- float[] hh = new float[m + 1];// hidden unit summation
+ float[] hh = new float[m + 1]; // hidden unit summation
hh[0] = 1;
for (i = 0; i < m; i++) {
double h = 0;
diff --git a/tika-core/src/main/java/org/apache/tika/detect/NNTrainedModelBuilder.java b/tika-core/src/main/java/org/apache/tika/detect/NNTrainedModelBuilder.java
index 9b4eab3854..f710a9d5ec 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/NNTrainedModelBuilder.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/NNTrainedModelBuilder.java
@@ -15,7 +15,6 @@
* limitations under the License.
*/
-
package org.apache.tika.detect;
import org.apache.tika.mime.MediaType;
diff --git a/tika-core/src/main/java/org/apache/tika/detect/NameDetector.java b/tika-core/src/main/java/org/apache/tika/detect/NameDetector.java
index 36d01e1711..2f49ec31ee 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/NameDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/NameDetector.java
@@ -23,37 +23,33 @@
import java.net.URLDecoder;
import java.util.Map;
import java.util.regex.Pattern;
-
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
/**
- * Content type detection based on the resource name. An instance of this
- * class contains a set of regular expression patterns that are matched
- * against the resource name potentially given as a part of the input metadata.
- * application/octet-stream.
- *
+ * application/octet-stream.
+ *
+ *
- *
- * application/octet-stream
*/
@@ -147,5 +134,4 @@ public MediaType detect(InputStream input, Metadata metadata) {
return MediaType.OCTET_STREAM;
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/NonDetectingEncodingDetector.java b/tika-core/src/main/java/org/apache/tika/detect/NonDetectingEncodingDetector.java
index 896a795318..a473cffaff 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/NonDetectingEncodingDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/NonDetectingEncodingDetector.java
@@ -21,24 +21,17 @@
import java.io.InputStream;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
-
import org.apache.tika.config.Field;
import org.apache.tika.metadata.Metadata;
-/**
- * Always returns the charset passed in via the initializer
- */
+/** Always returns the charset passed in via the initializer */
public class NonDetectingEncodingDetector implements EncodingDetector {
- //would have preferred final, but need mutability for
- //loading via TikaConfig
+ // would have preferred final, but need mutability for
+ // loading via TikaConfig
private Charset charset = StandardCharsets.UTF_8;
- /**
- * Sets charset to UTF-8.
- */
- public NonDetectingEncodingDetector() {
-
- }
+ /** Sets charset to UTF-8. */
+ public NonDetectingEncodingDetector() {}
public NonDetectingEncodingDetector(Charset charset) {
this.charset = charset;
diff --git a/tika-core/src/main/java/org/apache/tika/detect/OverrideDetector.java b/tika-core/src/main/java/org/apache/tika/detect/OverrideDetector.java
index b6c5a41f8a..ee2944d227 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/OverrideDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/OverrideDetector.java
@@ -18,17 +18,16 @@
import java.io.IOException;
import java.io.InputStream;
-
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
/**
- * Use this to force a content type detection via the
- * {@link TikaCoreProperties#CONTENT_TYPE_USER_OVERRIDE} key in the metadata object.
- * true then that byte is very unlikely to occur
- * in a plain text document.
- *
* +-------------------------+
* | Binary data byte ranges |
@@ -86,29 +79,29 @@ public class TextDetector implements Detector {
private final int bytesToTest;
/**
- * Constructs a {@link TextDetector} which will look at the default number
- * of bytes from the beginning of the document.
+ * Constructs a {@link TextDetector} which will look at the default number of bytes from the
+ * beginning of the document.
*/
public TextDetector() {
this(DEFAULT_NUMBER_OF_BYTES_TO_TEST);
}
/**
- * Constructs a {@link TextDetector} which will look at a given number of
- * bytes from the beginning of the document.
+ * Constructs a {@link TextDetector} which will look at a given number of bytes from the
+ * beginning of the document.
*/
public TextDetector(int bytesToTest) {
this.bytesToTest = bytesToTest;
}
/**
- * Looks at the beginning of the document input stream to determine
- * whether the document is text or not.
+ * Looks at the beginning of the document input stream to determine whether the document is text
+ * or not.
*
- * @param input document input stream, or null
+ * @param input document input stream, or null
* @param metadata ignored
- * @return "text/plain" if the input stream suggest a text document,
- * "application/octet-stream" otherwise
+ * @return "text/plain" if the input stream suggest a text document, "application/octet-stream"
+ * otherwise
*/
public MediaType detect(InputStream input, Metadata metadata) throws IOException {
if (input == null) {
@@ -137,5 +130,4 @@ public MediaType detect(InputStream input, Metadata metadata) throws IOException
input.reset();
}
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/TextStatistics.java b/tika-core/src/main/java/org/apache/tika/detect/TextStatistics.java
index 50f8d790aa..29252ccd67 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/TextStatistics.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/TextStatistics.java
@@ -35,11 +35,11 @@ public void addData(byte[] buffer, int offset, int length) {
}
/**
- * Checks whether at least one byte was seen and that the bytes that
- * were seen were mostly plain text (i.e. < 2% control, > 90% ASCII range).
+ * Checks whether at least one byte was seen and that the bytes that were seen were mostly plain
+ * text (i.e. < 2% control, > 90% ASCII range).
*
- * @return true if the seen bytes were mostly safe ASCII,
- * false otherwise
+ * @return true if the seen bytes were mostly safe ASCII, false
+ * otherwise
* @see TIKA-483
* @see TIKA-688
*/
@@ -53,8 +53,7 @@ public boolean isMostlyAscii() {
/**
* Checks whether the observed byte stream looks like UTF-8 encoded text.
*
- * @return true if the seen bytes look like UTF-8,
- * false otherwise
+ * @return true if the seen bytes look like UTF-8, false otherwise
* @since Apache Tika 1.3
*/
public boolean looksLikeUTF8() {
@@ -63,16 +62,18 @@ public boolean looksLikeUTF8() {
int safe = countSafeControl();
int expectedContinuation = 0;
- int[] leading = new int[]{count(0xc0, 0xe0), count(0xe0, 0xf0), count(0xf0, 0xf8)};
+ int[] leading = new int[] {count(0xc0, 0xe0), count(0xe0, 0xf0), count(0xf0, 0xf8)};
for (int i = 0; i < leading.length; i++) {
utf8 += leading[i];
expectedContinuation += (i + 1) * leading[i];
}
int continuation = count(0x80, 0xc0);
- return utf8 > 0 && continuation <= expectedContinuation &&
- continuation >= expectedContinuation - 3 && count(0xf8, 0x100) == 0 &&
- (control - safe) * 100 < utf8 * 2;
+ return utf8 > 0
+ && continuation <= expectedContinuation
+ && continuation >= expectedContinuation - 3
+ && count(0xf8, 0x100) == 0
+ && (control - safe) * 100 < utf8 * 2;
}
/**
@@ -95,13 +96,13 @@ public int count(int b) {
}
/**
- * Counts control characters (i.e. < 0x20, excluding tab, CR, LF,
- * page feed and escape).
- *
* +-------------------------+
* | Binary data byte ranges |
@@ -149,8 +150,10 @@ private int count(int from, int to) {
}
private int countSafeControl() {
- return count('\t') + count('\n') + count('\r') // tab, LF, CR
- + count(0x0c) + count(0x1b); // new page, escape
+ return count('\t')
+ + count('\n')
+ + count('\r') // tab, LF, CR
+ + count(0x0c)
+ + count(0x1b); // new page, escape
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/TrainedModel.java b/tika-core/src/main/java/org/apache/tika/detect/TrainedModel.java
index 0111b233bb..6d725ee80f 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/TrainedModel.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/TrainedModel.java
@@ -16,7 +16,6 @@
*/
package org.apache.tika.detect;
-
public abstract class TrainedModel {
public abstract double predict(double[] input);
diff --git a/tika-core/src/main/java/org/apache/tika/detect/TrainedModelDetector.java b/tika-core/src/main/java/org/apache/tika/detect/TrainedModelDetector.java
index 25b9f085be..170ed0ba02 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/TrainedModelDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/TrainedModelDetector.java
@@ -31,7 +31,6 @@
import java.nio.file.Path;
import java.util.HashMap;
import java.util.Map;
-
import org.apache.tika.io.TemporaryResources;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
diff --git a/tika-core/src/main/java/org/apache/tika/detect/TypeDetector.java b/tika-core/src/main/java/org/apache/tika/detect/TypeDetector.java
index 60d75c7b0c..d10ef00242 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/TypeDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/TypeDetector.java
@@ -17,27 +17,25 @@
package org.apache.tika.detect;
import java.io.InputStream;
-
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
/**
- * Content type detection based on a content type hint. This detector simply
- * trusts any valid content type hint given in the input metadata, and returns
- * that as the likely type of the input document.
+ * Content type detection based on a content type hint. This detector simply trusts any valid
+ * content type hint given in the input metadata, and returns that as the likely type of the input
+ * document.
*
* @since Apache Tika 0.3
*/
public class TypeDetector implements Detector {
/**
- * Detects the content type of an input document based on a type hint
- * given in the input metadata. The CONTENT_TYPE attribute of the given
- * input metadata is expected to contain the type of the input document.
- * If that attribute exists and contains a valid type name, then that
- * type is returned.
+ * Detects the content type of an input document based on a type hint given in the input
+ * metadata. The CONTENT_TYPE attribute of the given input metadata is expected to contain the
+ * type of the input document. If that attribute exists and contains a valid type name, then
+ * that type is returned.
*
- * @param input ignored
+ * @param input ignored
* @param metadata input metadata, possibly with a CONTENT_TYPE value
* @return detected media type, or application/octet-stream
*/
@@ -52,5 +50,4 @@ public MediaType detect(InputStream input, Metadata metadata) {
}
return MediaType.OCTET_STREAM;
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/XmlRootExtractor.java b/tika-core/src/main/java/org/apache/tika/detect/XmlRootExtractor.java
index 94d8531498..83f73e1ec7 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/XmlRootExtractor.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/XmlRootExtractor.java
@@ -20,19 +20,17 @@
import java.io.InputStream;
import java.util.Arrays;
import javax.xml.namespace.QName;
-
import org.apache.commons.io.input.CloseShieldInputStream;
import org.apache.commons.io.input.UnsynchronizedByteArrayInputStream;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.utils.XMLReaderUtils;
import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;
-import org.apache.tika.parser.ParseContext;
-import org.apache.tika.utils.XMLReaderUtils;
-
/**
- * Utility class that uses a {@link javax.xml.parsers.SAXParser} to determine
- * the namespace URI and local name of the root element of an XML file.
+ * Utility class that uses a {@link javax.xml.parsers.SAXParser} to determine the namespace URI and
+ * local name of the root element of an XML file.
*
* @since Apache Tika 0.4
*/
@@ -66,17 +64,17 @@ public QName extractRootElement(byte[] data) {
public QName extractRootElement(InputStream stream) {
return extractRootElement(stream, false);
}
-
+
private QName extractRootElement(InputStream stream, boolean throwMalformed) {
ExtractorHandler handler = new ExtractorHandler();
try {
- XMLReaderUtils.parseSAX(CloseShieldInputStream.wrap(stream),
- handler, EMPTY_CONTEXT);
+ XMLReaderUtils.parseSAX(CloseShieldInputStream.wrap(stream), handler, EMPTY_CONTEXT);
} catch (SecurityException e) {
throw e;
} catch (Exception e) {
- if (throwMalformed && (e instanceof CharConversionException
- || e.getCause() instanceof CharConversionException)) {
+ if (throwMalformed
+ && (e instanceof CharConversionException
+ || e.getCause() instanceof CharConversionException)) {
throw new MalformedCharException(e);
}
}
@@ -93,7 +91,6 @@ public void startElement(String uri, String local, String name, Attributes attri
this.rootElement = new QName(uri, local);
throw new SAXException("Aborting: root element received");
}
-
}
private static class MalformedCharException extends RuntimeException {
@@ -101,7 +98,5 @@ private static class MalformedCharException extends RuntimeException {
public MalformedCharException(Exception e) {
super(e);
}
-
}
-
}
diff --git a/tika-core/src/main/java/org/apache/tika/detect/ZeroSizeFileDetector.java b/tika-core/src/main/java/org/apache/tika/detect/ZeroSizeFileDetector.java
index 5ce52681e2..86f4917250 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/ZeroSizeFileDetector.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/ZeroSizeFileDetector.java
@@ -18,13 +18,10 @@
import java.io.IOException;
import java.io.InputStream;
-
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
-/**
- * Detector to identify zero length files as application/x-zerovalue
- */
+/** Detector to identify zero length files as application/x-zerovalue */
public class ZeroSizeFileDetector implements Detector {
public MediaType detect(InputStream stream, Metadata metadata) throws IOException {
if (stream != null) {
diff --git a/tika-core/src/main/java/org/apache/tika/detect/package-info.java b/tika-core/src/main/java/org/apache/tika/detect/package-info.java
index dede49cfb6..b04a9d1c96 100644
--- a/tika-core/src/main/java/org/apache/tika/detect/package-info.java
+++ b/tika-core/src/main/java/org/apache/tika/detect/package-info.java
@@ -15,8 +15,6 @@
* limitations under the License.
*/
-/**
- * Media type detection.
- */
+/** Media type detection. */
@aQute.bnd.annotation.Version("1.0.0")
package org.apache.tika.detect;
diff --git a/tika-core/src/main/java/org/apache/tika/embedder/Embedder.java b/tika-core/src/main/java/org/apache/tika/embedder/Embedder.java
index 2af59d326e..d1bead9eb0 100644
--- a/tika-core/src/main/java/org/apache/tika/embedder/Embedder.java
+++ b/tika-core/src/main/java/org/apache/tika/embedder/Embedder.java
@@ -21,7 +21,6 @@
import java.io.OutputStream;
import java.io.Serializable;
import java.util.Set;
-
import org.apache.tika.exception.TikaException;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
@@ -36,11 +35,11 @@
public interface Embedder extends Serializable {
/**
- * Returns the set of media types supported by this embedder when used with
- * the given parse context.
- *
- *
*
- * @param metadata document metadata (input and output)
+ * @param metadata document metadata (input and output)
* @param originalStream the document stream (input)
- * @param outputStream the output stream to write the metadata embedded data to
- * @param context parse context
- * @throws IOException if the document stream could not be read
+ * @param outputStream the output stream to write the metadata embedded data to
+ * @param context parse context
+ * @throws IOException if the document stream could not be read
* @throws TikaException if the document could not be parsed
*/
- void embed(Metadata metadata, InputStream originalStream, OutputStream outputStream,
- ParseContext context) throws IOException, TikaException;
-
+ void embed(
+ Metadata metadata,
+ InputStream originalStream,
+ OutputStream outputStream,
+ ParseContext context)
+ throws IOException, TikaException;
}
diff --git a/tika-core/src/main/java/org/apache/tika/embedder/ExternalEmbedder.java b/tika-core/src/main/java/org/apache/tika/embedder/ExternalEmbedder.java
index b2411d7a89..a1b2cf65c1 100644
--- a/tika-core/src/main/java/org/apache/tika/embedder/ExternalEmbedder.java
+++ b/tika-core/src/main/java/org/apache/tika/embedder/ExternalEmbedder.java
@@ -29,10 +29,8 @@
import java.util.List;
import java.util.Map;
import java.util.Set;
-
import org.apache.commons.io.IOUtils;
import org.apache.commons.io.output.UnsynchronizedByteArrayOutputStream;
-
import org.apache.tika.exception.TikaException;
import org.apache.tika.io.TemporaryResources;
import org.apache.tika.io.TikaInputStream;
@@ -43,50 +41,49 @@
import org.apache.tika.parser.external.ExternalParser;
/**
- * Embedder that uses an external program (like sed or exiftool) to embed text
- * content and metadata into a given document.
+ * Embedder that uses an external program (like sed or exiftool) to embed text content and metadata
+ * into a given document.
*
* @since Apache Tika 1.3
*/
public class ExternalEmbedder implements Embedder {
- /**
- * Token to be replaced with a String array of metadata assignment command
- * arguments
- */
+ /** Token to be replaced with a String array of metadata assignment command arguments */
public static final String METADATA_COMMAND_ARGUMENTS_TOKEN = "${METADATA}";
- /**
- * Token to be replaced with a String array of metadata assignment command
- * arguments
- */
+
+ /** Token to be replaced with a String array of metadata assignment command arguments */
public static final String METADATA_COMMAND_ARGUMENTS_SERIALIZED_TOKEN =
"${METADATA_SERIALIZED}";
+
private static final long serialVersionUID = -2828829275642475697L;
private final TemporaryResources tmp = new TemporaryResources();
- /**
- * Media types supported by the external program.
- */
+
+ /** Media types supported by the external program. */
private Setmetadata.
+ * Constructs a collection of command line arguments responsible for setting individual metadata
+ * fields based on the given metadata.
*
* @param metadata the metadata to embed
* @return the metadata-related command line arguments
@@ -303,18 +290,20 @@ protected List