Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure cache dir #1712

Merged
merged 3 commits into from
Dec 17, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ Configuration.rootFile=grpc-api.prop
workspace=.
devroot=.

deephaven.console.type=groovy
deephaven.console.type=groovy
deephaven.cache.dir=/cache
2 changes: 2 additions & 0 deletions docker/server/src/main/configure/image-bootstrap.properties
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Configuration.rootFile=grpc-api.prop
workspace=.
devroot=.

deephaven.cache.dir=/cache
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
import io.deephaven.api.util.NameValidator;
import io.deephaven.base.FileUtils;
import io.deephaven.compilertools.CompilerTools;
import io.deephaven.configuration.Configuration;
import io.deephaven.engine.table.Table;
import io.deephaven.engine.table.TableDefinition;
import io.deephaven.engine.table.lang.QueryLibrary;
Expand All @@ -31,11 +30,11 @@
* evaluateScript which handles liveness and diffs in a consistent way.
*/
public abstract class AbstractScriptSession extends LivenessScope implements ScriptSession, VariableProvider {
public static final String CLASS_CACHE_LOCATION = Configuration.getInstance()
.getStringWithDefault("ScriptSession.classCacheDirectory", "/tmp/dh_class_cache");

private static final Path CLASS_CACHE_LOCATION = CacheDir.get().resolve("script-session-classes");

public static void createScriptCache() {
final File classCacheDirectory = new File(CLASS_CACHE_LOCATION);
final File classCacheDirectory = CLASS_CACHE_LOCATION.toFile();
createOrClearDirectory(classCacheDirectory);
}

Expand All @@ -60,8 +59,9 @@ private static void createOrClearDirectory(final File directory) {
protected AbstractScriptSession(@Nullable Listener changeListener, boolean isDefaultScriptSession) {
this.changeListener = changeListener;

// TODO(deephaven-core#1713): Introduce instance-id concept
final UUID scriptCacheId = UuidCreator.getRandomBased();
classCacheDirectory = new File(CLASS_CACHE_LOCATION, UuidCreator.toString(scriptCacheId));
classCacheDirectory = CLASS_CACHE_LOCATION.resolve(UuidCreator.toString(scriptCacheId)).toFile();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed introducing props:
INSTANCE_ID (if set, this is the ID)
INSTANCE_ID_FILE (if set and INSTANCE_ID isn't this is the file we load from or generate)
else, we randomly allocate an instance ID and be sure to clean up on exit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Punting - #1713

createOrClearDirectory(classCacheDirectory);

queryScope = newQueryScope();
Expand Down
39 changes: 39 additions & 0 deletions engine/table/src/main/java/io/deephaven/engine/util/CacheDir.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package io.deephaven.engine.util;

import io.deephaven.configuration.Configuration;

import java.nio.file.Path;

/**
* The cache directory is a directory that the application may use for storing data with "cache-like" semantics.
* Cache-like data is data that may be preserved across restarts, but the application logic should not make the
* assumptions that the data will be available.
*/
public final class CacheDir {
private static final String DEEPHAVEN_CACHE_DIR = "deephaven.cache.dir";
private static final String JAVA_IO_TMPDIR = "java.io.tmpdir";

private static final Path cacheDir;

static {
final Configuration config = Configuration.getInstance();
if (config.hasProperty(DEEPHAVEN_CACHE_DIR)) {
cacheDir = Path.of(config.getProperty(DEEPHAVEN_CACHE_DIR));
} else {
cacheDir = Path.of(System.getProperty(JAVA_IO_TMPDIR), "deephaven", "cache");
}
}

/**
* Return the value for the configuration {@value DEEPHAVEN_CACHE_DIR} if it is present.
*
* <p>
* Otherwise, return "%s/deephaven/cache", parameterized from the value of the system property
* {@value JAVA_IO_TMPDIR}.
*
* @return the cache dir
*/
public static Path get() {
return cacheDir;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ private void load(final String businessCalendarConfig)
final File calendarFile = inputStreamToFile(inputStream);
final BusinessCalendar businessCalendar = DefaultBusinessCalendar.getInstance(calendarFile);
addCalendar(businessCalendar);
calendarFile.deleteOnExit();
calendarFile.delete();
} else {
logger.warn("Could not open " + filePath + " from classpath");
throw new RuntimeException("Could not open " + filePath + " from classpath");
Expand Down