Resolving doesn't work

[markehammons]

I have a config like so:

env_name=local
installation_directory_path=${exePath}/tmp/test_local_docker
#me_user=${whoami}

docker {
    network=test_env
    internal_storage=1
    default_mount=false
    persistent_data=false
}

windows_env=false

clickhouse {
    global {
        replication_factor=1
        image_file=clickhouse-server
        image_name=yandex/clickhouse-server
        container_name=local_clickhouse_server
    }

    mapping {
        http_port=8223
        tcp_port=9101
        interserver_http_port=9101
        docker {
            data_path=/clickhouse/clickhouse-data
            logs_path=/clickhouse/clickhouse-logs
            file_path=/clickhouse/clickhouse-files
        }
        host {
            data_path=${installation_directory_path}/clickhouse-data
            logs_path=${installation_directory_path}/clickhouse-logs
            file_path=${installation_directory_path}/clickhouse-files
        }
        max_server_memory_usage_to_ram_ratio=0.2
    }
}

I merge it with a programmatically generated config

exePath=/home/mhammons/Documents/Projects/opsScala/core/target/scala-2.13

and call resolve, but no matter what I do I get the following error:

org.ekrich.config.ConfigException$UnresolvedSubstitution: String: 102: Could not resolve substitution to a value: ${installation_directory_path}
        at java.lang.Throwable.fillInStackTrace(Unknown Source)
        at org.ekrich.config.ConfigException$UnresolvedSubstitution.<init>(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.ConfigConcatenation.$anonfun$resolveSubstitutions$2(Unknown Source)
        at org.ekrich.config.impl.ConfigConcatenation$$Lambda$2.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.ConfigConcatenation.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext$.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolveWith(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolveWith(Unknown Source)
        at ops.environments.Local$.<init>(Unknown Source)
        at <none>._SM23ops.environments.Local$G4load(Unknown Source)
        at ops.environments.package$.<init>(Unknown Source)
        at <none>._SM25ops.environments.package$G4load(Unknown Source)
        at ops.Ops$.<init>(Unknown Source)
        at <none>._SM8ops.Ops$G4load(Unknown Source)
        at <none>.main(Unknown Source)
        at <none>.__libc_start_main(Unknown Source)
        at <none>._start(Unknown Source)
        ```

[ekrich]

Thank-you for trying out the library. Sorry you are not finding the library working as expected. I am not much of an expert on using the library so if you could provide some minimized code so I could test it out that would really help.

Another option is to try the lightbend config library with the JVM to see if it works in that environment. I have not been able to follow all the changes they have made so it could be fixed in that library but not this one.

[markehammons]

env=local
basedOnEnv=${env}

This is a minimal example. Loaded with

  val config2 = {
    ConfigFactory
      .parseString(
        new String(
          Files.readAllBytes(filesPath.resolve("parameters_base.conf"))
        )
      )
      .resolve()
  }

  println(config2.getString("basedOnEnv"))

Stack trace is:

org.ekrich.config.ConfigException$NotResolved: need to Config#resolve(), see the API docs for Config#resolve(); substitution not resolved: ConfigReference(${env})
        at java.lang.Throwable.fillInStackTrace(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.notResolved(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.valueType(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.valueType(Unknown Source)
        at org.ekrich.config.impl.AbstractConfigValue.equals(Unknown Source)
        at java.util.Objects$.equals(Unknown Source)
        at java.util.AbstractCollection.$anonfun$contains$1(Unknown Source)
        at java.util.AbstractCollection$$Lambda$1.apply(Unknown Source)
        at java.util.ScalaOps$JavaIteratorOps$.exists$extension(Unknown Source)
        at java.util.ScalaOps$JavaIterableOps$.exists$extension(Unknown Source)
        at java.util.AbstractCollection.contains(Unknown Source)
        at java.util.Collections$WrappedCollection.contains(Unknown Source)
        at java.util.Collections$$anon$22.contains(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext$.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolveWith(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolve(Unknown Source)
        at ops.environments.Local$.<init>(Unknown Source)
        at <none>._SM23ops.environments.Local$G4load(Unknown Source)
        at ops.environments.package$.<init>(Unknown Source)
        at <none>._SM25ops.environments.package$G4load(Unknown Source)
        at ops.Ops$.<init>(Unknown Source)
        at <none>._SM8ops.Ops$G4load(Unknown Source)
        at <none>.main(Unknown Source)
        at <none>.__libc_start_main(Unknown Source)
        at <none>._start(Unknown Source)

Another option is to try the lightbend config library with the JVM to see if it works in that environment. I have not been able to follow all the changes they have made so it could be fixed in that library but not this one.

I can try that tomorrow if you'd like, but I imagine resolving works there

[ekrich]

@markehammons Update: while porting tests to Scala Native, I ran across this that failed only only Scala Native.

[error] Test org.ekrich.config.impl.PropertiesTest.makeListWithNumericKeysAndMerge failed: org.ekrich.config.ConfigException$UnresolvedSubstitution: String: 5: Could not resolve substitution to a value: ${a} was part of a cycle of substitutions involving ${a}, ${a}, took 0.004 sec

[ekrich]

@markehammons If you are following, I determined the issue. java.util.IdentityHashMap needed fixing on Scala Native. Luckily I created a good version on Scala.js that I could port over. I put in a pull request there. scala-native/scala-native#2425

We will have to wait for a new release of Scala Native and then this PR where I extend the code coverage can be merged and then we can release another version of sconfig. Kind of a long path but good either way. #205

[markehammons]

glad to hear it. I haven't been using scala-native much (mainly because scala 3 has spoiled me), but I'm looking forward to its changes, and hopefully the project I'm currently working on can be used to bridge between scala native and scala on the jvm.

…

On Sat, Oct 30, 2021 at 3:40 AM Eric K Richardson ***@***.***> wrote: @markehammons <https://github.com/markehammons> If you are following, I determined the issue. java.util.IdentityHashMap needed fixing on Scala Native. Luckily I created a good version on Scala.js that I could port over. I put in a pull request there. scala-native/scala-native#2425 <scala-native/scala-native#2425> We will have to wait for a new release of Scala Native and then this PR where I extend the code coverage can be merged and then we can release another version of sconfig. Kind of a long path but good either way. #205 <#205> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#141 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAT5CP3K3KPBL5KCAD2GWQLUJNEIDANCNFSM4WVF6TEA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[ekrich]

Closed via this PR: #205

So using Scala Native 0.4.2 and sconfig 1.4.6 should resolve the issue.