Fix #1843 (java.util.zip.ZipException in HttpZipLocator) #1842
Conversation
…r back ZipExceptions.
Ali-RS
changed the title
Ali-RS
changed the title
|
Hmm, looking at the specs though... https://en.wikipedia.org/wiki/ZIP_(file_format) It reads this from the central directory file header:
Adds the local file header size and the file name size (variable length) and then the extra field size (variable length). But probably the extra field size here now just refers to the Central Directory File Header extra field. Right? This is not clear. This is either correct statement or what is happening here is that the ZIP file is just not in the correct form. What I think is that this is broken. And this fix wont fix it. What I think is the correct solution is that we should just always store the local file header offset. And start the reading from there, the header, since clearly it is of variable size and the variable size can't be figured out from the central directory. Then of course we would need to read a bit extra again like with the end header. Or sort the entries found and figure out the size of file data + file header from subsequent entries. |
|
Maybe clearer example of ZIP format: |
Right, this is what it says in source: and it is used to calculate the new offset for reading the next entry: |
|
Taking a quick look at the Wikipedia link I see there are local file name length and local extra field length as well. seems they are not used anywhere in HttpZipLocator:thinking: Are they perhaps meant to be added to local offset instead of the central directory headers CENNAM (
I do not know the internals of zip format so just guessing. |
Yeah, it doesn't read the local file header. That is my point. It guesses the length of the local file header from the central directory header and that is the bug. But you can't use those LOCNAM in the central directory header of course. Reading of the local file header is missing from the code totally currently as the length of it is thought to be known from the central directory header. But that is false. So what I am proposing that we just start to read from the local file header always. Meaning in code: So what we then have in the hash map, is the starting location for each entry. Starting location to the local file header of the entry to be exact. Then we make a guess for reading the file data from the HTTP connection. What we want to read is ZipEntry.LOCHDR (size of the local file header static parts) + nameLen (from the central directory header, assume it is always the same in both) + file data length (from the central directory header) + GUESS the length of the extra field (given by LOCEXT from the local header). This way we read a bit extra, and future bug maybe that someone wrote 7GB to the extra header. I would guess that initially 100 bytes extra is enough for a guess. And shouldn't break the idea of the optimization, hopefully. I would imagine assets are big, 100 bytes is nothing in comparison. |
|
I can also code this in if you agree with me in this approach and you don't want to take a stab at this. In addition:
|
|
Note that JME has a very long history of "not invented here" aversion style development... whether because the developer was unaware of alternatives (even in the JDK) or because of some specific requirement they had latched onto. I do not know the history or requirements for this class (though I can guess on the latter) but it might be worth exploring whether "modern Java" can already do what this class is trying to do. (And by "modern", I mean post Java 1 point 2 (written out for emphasis how old I mean)). For example, Java (effectively forever) has supported URL jar references to specific files within the jar... and because it uses this for remote class loading, it's been pretty well optimized. Meaning: we might be able to rip out all of this weird special code and convert it to some clever URL construction or maybe just create a URLClassLoader that points to the zip file and then use it to resolve resources like a regular classpath asset locator does. |
Please feel free to submit your change. But please note that Java 11 HttpClient still is not supported on android (I have made a request to support it) so better be avoided if we want android compatibility.
There is ZipInputStream in java (since java 1.1) that does what HttpZipLocator is doing but seems the issue is that for loading a specific ZipEntry we must traverse through all other entries using just for ref: https://itecnote.com/tecnote/java-how-to-extract-a-single-file-from-a-remote-archive-file/
Hmm, that sounds like a very interesting idea. Java does already handle loading class from a remote jar like so we may be able to make use of it to handle resources from remote zip like |
Damn Android, why you have to be different... Let's forget the HttpClient then.
I came to that conclusion that if we use anything ready, we will lose the optimization of just downloading the parts that we are interested in. And that seems to be the whole point in the class. Surely Internet speeds are probably faster today, but so are asset sizes probably on average bigger. Although not supporting Zip64, we have a limit on 4Gb ZIP file. This locator has very niche usage probably, like you expressed in the forum. So I'd just fix the thing, keeping the optimization in place even though it is the part causing all the headaches :D |
|
But I would think that a Java URLClassLoader accessing a jar through a URL would have to have solved all of these problems and used to be super common. It has to jump all over the place loading one .class file or another .class file... all over the jar. So either it's doing clever caching or something even more clever. My point is, don't necessarily trust that the "old ones" were smarter than we are. If you look through JME history then you can find plenty of cases where the original authors just didn't know any better... or on that day just thought it would be easier to roll their own than to learn how to do it a built-in way. I also think that in modern Java usage, using this class is a bad idea in general. Without any real modern environments where "all of the code and data" is on a web site (ie: Applets), this is going to be poor experience for the user over any alternative. It's tough to decide what the proper use-case would be. And without some use-cases, can we honestly say that the four-line URLClassLoader version would perform worse than our bodged together fragile code? |
|
Well, I mean.. after the fix it wouldn't been that fragile, I'm super confident about that :D ZIP format is quite simple and doesn't change. I agree that the code is really complicated for what it is trying to do. Just because it doesn't want to download the whole ZIP file, just the part needed. I can try it hopefully today, but I'm quite confident that URLClassLoader approach will download the whole file. It probably caches it. At least the API faintly suggests that there are caches but you never know without looking the code more clearly what cache they are referring to (could be just the HTTP headers where it declares that no cache plz). But do we call this a success then? That the whole file is downloaded once, is that acceptable? That will be a change of behavior and might be noticeable if someone has that kind of weird use case where somehow only few assets are required. If it downloads the ZIP more than once, then I would at least argue that this would be too big of a change to fix this bug (or to introduce in general). |
|
If it does download the whole file then I maybe agree. But I have doubts. I'm old enough to have lived through the age of the ubiquitous Java applet and we somehow didn't have to wait 5-10 minutes every time we started one up. When we assume how it's going to work so continue hand-rolling then we do exactly what the original authors did. Which is fine... sometimes that's how things get done. Just a little unfortunate. |
@tonihele something like this? The above code works successfully. The thing to note is that LOCNAM and CENNAM are the same lengths and LOCEXT is 0. I guess this is true for 99% (?) of the case when we create a zip using an archive creation program unless someone manually adds a local extra field. i.e. we could ignore the local header and consider the local "extraLength = 0" but I like your approach more as it is guaranteed to work in all case. Please let me know if this code looks good to you, then I will update my PR. |
|
That looks correct yes. I would also guess that the name is always the same, whether you got it from the central or the local. But you better make sure and this is the correct way to do it. What comes now to the guessing how much we need to read. I would say we are safe with 100 bytes extra, ZIP64 for example adds just 32 bytes to the local headers, in form of the extra headers. Surely we wont support it currently and well... And the name should be the same.. This is probably just fine. What I also noted that the whole class... I guess it can be stateful, I have seen this a lot in jME. But there might be a risk with multithreading, although I'm not sure how the AssetManager handles the requests, does it allow for multithreaded processing or not. So there is this:
And lot of those streams could be handled with |
|
Of course the cleaning up and all that would be better of as its own PR, but this is just... How could I put it, to me it doesn't matter that much. I'm always guilty of combining formatting, fix and optimizations into same PR. Which is horrible I know. I'm quite curious to add ZIP 64 support. For absolutely no reason at all :D |
Sorry for my ignorance but not sure if I am understanding this. What are these 100 bytes for? |
From the central directory you get the offset where the local header starts and the file size. But you don't exactly know how much of the ZIP you should read as the local header size varies. HTTP is expensive, relatively. So we probably want to read the file at one go (like it is already + the central directory reading of course (200 bytes from the end)). Thus we need to make a guess what is the range of bytes we need. The formula to calculate the range of bytes to read is: GUESS could be 100 bytes. If extra header size is 0, like it seems to be in many of the cases you tried. We will of course read extra 100 bytes. |
Hmm, but we already know the extra fields length (obtained from LOCEXT field) so why need to guess it? See https://docs.fileformat.com/compression/zip/#local-file-header |
|
Yeah, you are reading it from there yes. You know it AFTER you have downloaded the file. The asset requested.
The number 3 dissected:
To download the asset you need to either take the guess route. Or you need to download in 1-2 parts once you have read the local header. This is how I understand it works currently too. And that is the bug. It guesses the file starting location from the central directory. This fails because the file starting location is depending on the file local header length, which is unknown. It can only be guessed. It is currently guessed, and I suggested making a better guess. But still a guess. To not take a guess you could download first with the original guess (minus the extra field length like your PR originally did, since the extra field length is from the wrong header, |
If you take a look at the code I showed above, I'm reading it in 2 steps, in the first connection, I just read the local header to obtain the name length and extra fields length then close the connection, now I already know from where the file data starts ( fileDataOffset = entry.offset + ZipEntry.LOCHDR + nameLength + extraLength ) then I make a new connection to read the file data. |
Ah yes sorry, that is the route where you don't need guesses yes. And I'm fine with having 2 connections per file. It is less confusing than further trying to optimize like I described with the guesses and all. |
… calculate file data offset.
It is used from a static method |
|
Probably just recreate the decoder every time. JME bends over so far backwards to avoid creating objects that it ends up breaking its spine and having its head up its own... you get the picture. And yes, quite often this leads to major and often-subtle threading bugs (see my previous issues about animation tracks). Unless we have evidence that creating a new decoder will be expensive then just create one when needed. |
|
Yeah, I would say too that re-create it. It is the safest option. That can't be the heaviest operation there... Then you can still have the method as static. |
|
What about these: Should they be non-static as well? They are also used in |
|
Yeah, seems like a threading problem. |
|
Then I have to make |
|
I'm done with the update. Please feel free to review the changes. |
|
I think it is excellent! Good work! |
|
Thanks! I could not do it without your help, guys. |
|
Unless there's something to discuss, I plan to self-integrate this PR in about 24 hours. |
* Android: Implemented AndroidNativeBufferAllocator - Deprecated AndroidBufferAllocator (jMonkeyEngine#1821) * [skip ci] update natives snapshot * Add GL debug capabilities (jMonkeyEngine#1790) * Add GL debug capabilities * Fix: check for null names * Add java types to VarType and type checks to MatParam (jMonkeyEngine#1797) * SettingsDialog: Fixed LAF * SettingsDialog: Updated jme3 copyright * README.md: add a link to Chatter Games website * Update README.md to include Exotic Matter (jMonkeyEngine#1838) Would be great to have our game Exotic Matter also mentioned as its engine is based on jME. Thanks! * jme3-examples: update the fallback URLs for "TerrainGridTestData.zip" * Improved code readability: ParticlePointMesh and ParticleTriMesh (jMonkeyEngine#1831) * jme3-core: add tests for Transform.toString() * Fix issue jMonkeyEngine#1839 (Memory Leak in DefaultLightFilter) Co-authored-by: Lukas Habring <lukas@Lukas-PC> * Added getter & setter for FilterPostProcessor.depthFormat (jMonkeyEngine#1841) * Added setter for FilterPostProcessor.depthFormat * Added getter for FilterPostProcessor.depthFormat * update the Gradle wrapper to v7.5.1 * main.yml: GitHub Action's ubuntu-18.04 environment is deprecated * bugfix: mergedJavadoc task is incompatible with Gradle v7 * main.yml: udate "actions/setup-java" to v3 * main.yml: add JDK 17 tests * build.gradle: update gradleVersion in case user runs the "wrapper" task * Fix jMonkeyEngine#1843 (java.util.zip.ZipException in HttpZipLocator) (jMonkeyEngine#1842) * HttpZipLocator:fix invalid code lengths set & invalid distance too far back ZipExceptions. * Get file name length and extra field length from local file header to calculate file data offset. * Make fields `byteBuf`, `charBuf`, `utf8Decoder` non-static because they are not thread-safe. * Surround streams with try-with-resources block. * Clean up the display modes parsing * No need to separately test for the contains * README.md: SDK v3.4 has been released * Some enhancement to new animation system (jMonkeyEngine#1845) * Some enhancement to new animation system, including: * Option to enable/disable animation mask propagation to child actions. * Option to control max transition weight. For example useful for controlling smooth animation transition when an animation is removed from an upper layer. * Added animation loop support in AnimLayer. * AnimLayer can now also keep action name, so one can easily lookup currently playing action name in an specific layer. * Minor Javadoc fix. * AnimLayer: clear `currentActionName` inside `cloneFields` method. * Added a Loop tween to Tweens factory class (jMonkeyEngine#1846) * Added a Loop tween to Tweens factory class. Supports looping by count or duration. * Redesigned the Loop tween to work similar to Sequence tween. Now fast forwarding the loop will also try to catch up the loops left behind making sure they always see their 'length'. * Added the missing Override annotation. * Added Tweens.cycle() and Tweens.invert() methods (jMonkeyEngine#1849) Cycle is used for running delegate tween back and force and Invert is used to run delegate tween backward. * BlendAction: resolve slow-motion side effect caused by stretching actions (jMonkeyEngine#1848) * BlendAction: resolve slow motion side effect caused by stretching any action that doesn't have the same length. It generates speed factor for each child animation that are dynamically interpolated and applied to base speed based on the blend weight taken from blend space. * Add missing javadoc. * Add Copyright. * Renamed calculateSpeedFactors() to applyDefaultSpeedFactors() and made it non static. * Fix issue jMonkeyEngine#1850 (JmeSystem.writeImageFile() throw java.nio.BufferUnderflowException) (jMonkeyEngine#1851) * jme3-jogg: upgrade the j-ogg-all library to v1.0.2 * jMonkeyEngine#1569 Fix license file to be better detected by GitHub (jMonkeyEngine#1855) * jme3-plugins: update the "gson" library to v2.9.1 * workflows/main.yml: update the "checkout" action to v3 * fix: broken link in README.md (jMonkeyEngine#1858) Signed-off-by: Kasper Aaquist Johansen <kasperaaquist@gmail.com> Signed-off-by: Kasper Aaquist Johansen <kasperaaquist@gmail.com> * workflows/main.yml: update wrapper-validation-action to v1.0.5 * update wrapper-validation-action to v1.0.5 (one more place) * README.md: SDK v3.5.2 has now been published * Quaternion: javadoc * Add instance culling function in InstancedGeometry (jMonkeyEngine#1865) * Added a workaround to prevent shadow disappearing on instanced geometries away from camera by introducing an instance culling function on InstancedGeometry. There also is a default implementation provided. * Removed the bound-scale hack for shadow disappearing issue from DefaultInstanceCullingFunction. The “right” solution is to have it pay attention to the frustums of whatever shadow-casting lights are around… but anyway developers now can implement their own frustum culling however they like or even "unset" it to disable instance culling which should also resolve the shadow issue. * Remove unused imports. * Fix: make the stencil test functions usable. (jMonkeyEngine#1866) * Fix: make the stencil test functions usable. * Fix: formatting * Fix: formatting * Fix: formatting * Fix: copyright year * upgrade to Gradle v7.6 (for its Java 19 support) * Read shorts properly * More efficient logging * Read unsigned byte properly * Remove unnecessary byte and call the variable c as in the formulas * move SettingsDialog and ErrorDialog to new jme3-awt-dialogs module (jMonkeyEngine#1876) * Refactory Settings/Error dialogs in JmeDialogsFactory and jme3-awt-dialogs * add build.gradle * Add copyright headers * Fix formatting and documentation Co-authored-by: riccardobl <riccardo0blb@gmail.com> * JmeSurfaceView: Package migration (jMonkeyEngine#1819) * JmeSurfaceView: migration to new package (com.jme3.view.surfaceview) * JmeSurfaceView: migration to new package (com.jme3.view.surfaceview) * upgrade the groovy-test library to v3.0.13 * jme3-core: correct/clarify javadoc * Implementation of a glTF extension loader for KHR_texture_transform (jMonkeyEngine#1869) * Implementation of a glTF extension loader for KHR_texture_transform * Thread-safe version of the glTF extension loader for KHR_texture_transform * Updated thread-safe version of the glTF extension loader for KHR_texture_transform * Fix (switched indices of the translation matrix): thread-safe version of the glTF extension loader for KHR_texture_transform * Added support for texCoord, fixed matrix comparison * Simplified matrix comparison, removed trailing whitespaces * Update to differentiate transformations applied to different UV sets * Improved memory usage for transformMap * Specified Map generic types & removed unnecessary cast Co-authored-by: Manuel <Manuel@DESKTOP-6RJH3UF> * jme3-core: test the com.jme3.math.Triangle class * bugfix: Mesh.getTriangle() may yield an incorrect centroid * Fix jMonkeyEngine#1867 (LightFilter gets applied even if not needed) (jMonkeyEngine#1872) * Add NullLightFilter.java * Add usage of null light filter when rendering shadowmaps * Fix formatting * Make static NullLightFilter final * Fix formatting and author * BlendableAction: Fix JavaDoc for setMaxTransitionWeight & replace assert with IllegalArgumentException (jMonkeyEngine#1881) * Fix jMonkeyEngine#1412 (GltfLoader does not support AO packed in MetallicRoughnessMap) (jMonkeyEngine#1880) * Fix jMonkeyEngine#1882 (J3MLoader always generates mips ignoring MinFilter) (jMonkeyEngine#1884) * Get texture mips generation flag from MinFilter specified in j3m file. * Update copyright date. * Fix J3MLoaderTest failing. * Fix TestMaterialWrite failing. * Update copyright date. * Use Trilinear if no min filter is specified in j3m file. * Add copyright note in J3MOutputCapsule. * Fix J3MLoaderTest failing. * Made extension loaders non-static to avoid concurrency issues (jMonkeyEngine#1886) Now each GltfLoader instantiated via ThreadLocal will have its own instances of extension loaders. * Fix jMonkeyEngine#1883 (Image class wrongly setting GL mips flags inside the constructor) (jMonkeyEngine#1885) * AreaUtils: Migrated package to `com.jme3.util` (jMonkeyEngine#1826) * AreaUtils: Migrated package to `com.jme3.util` * com.jme3.uitl.AreaUtils: removed code duplicates - added `final` class specifier * com.jme3.util.AreaUtils: fixed duplication build error * utils/AreaUtils.java: full migration to the utility package * scene/control/AreaUtils.java: removed utility methods - delegated functionality to `jme3.utils.AreaUtils` * scene/AreaUtils: fixed `jme3.util` package linking typo * Renderer: javadoc correction * update the groovy-test library to v3.0.14 * LICENSE.md: add 2023 to copyright years * Fix jMonkeyEngine#1892 (TestChooser does not show classes list when run with java 8) (jMonkeyEngine#1893) * TestChooser:fix class list not showing when run with java 8. * Update copyright date. * main.yml: use "temurin" openjdk. Fix jMonkeyEngine#1896 (jMonkeyEngine#1897) * README.md: add Demon Lord to the list of published games * Fix jMonkeyEngine#1773 (Wrong particle position when `worldSpace` flag equals to true) (jMonkeyEngine#1889) * Add test case for issue jMonkeyEngine#1773. * Fix wrong particle position when using 'EmitterMeshVertexShape' or 'EmitterMeshFaceShape' and worldSpace flag equal to true. The old code was interpolating particles position toward emitter world position and this was only working fine for EmitterPointShape and in the other shapes this was causing particles not keep the shape because they were being dragged toward emitter position. The new code calculates the distance vector from emitter last location to the current emitter location and subtracts it from particles position to generate a hypothetical position that is used for interpolation. * Add javadoc to TestIssue1773. * Minor javadoc fix. * jme3-niftygui: solve issue jMonkeyEngine#1891 (incorrect fullscreen layout) (jMonkeyEngine#1895) * jme3-lwjgl:updated to lwjgl v2.9.4 hosted by org.jmonkeyengine. Fix jMonkeyEngine#1247, jMonkeyEngine#1215, jMonkeyEngine#947 (jMonkeyEngine#1902) * Fix jMonkeyEngine#1890 (crashes attempting to run example apps in fullscreen with LWJGL v2) (jMonkeyEngine#1898) * jme3-lwjgl:fallback to standard 60Hz fullscreen display mode if the specified frequency in the AppSettings is not available and log a warning. * Fallback to whatever bps or frequency is available. Added a wild-card value to let selecting any bps or frequency available by passing -1. * Support AWT display frequency model. Looks like AWT uses mathematics round to convert float frequency values to int while lwjgl 2 uses mathematics floor. For example if frequency is 59.83, AWT will return 60 but lwjgl 2 will return 59. * Remove redundant check. * Added documentation for getFullscreenDisplayMode method. * When in VR attach the debug scene to the two eye's scenes. Fix#1795 (jMonkeyEngine#1888) * jMonkeyEngine#1795 When in VR attach the debug scene to the two eye's scenes This ensures they show up correctly * jMonkeyEngine#1795 Whitespace corrections * jMonkeyEngine#1795 Further whitespace corrections * jMonkeyEngine#1795 Yet more whitespace corrections * jMonkeyEngine#1795 Add explanatory comment as to why VR and non-VR have totally different approaches * Refactored PBR Terrain to use new for-loops. Fix jMonkeyEngine#1785 (jMonkeyEngine#1901) Drastically reduced code by utilizing JME's new potential for define-compatible for-loops in shaders. Also cleaned up some other little things like indentations, comments, and code placement so that the code is much more organized and easier to understand. * add the Spatial.addControlAt() method (jMonkeyEngine#1899) * JmeContext: add a getSystemListener() method (jMonkeyEngine#1894) * jme3-examples: add tests for issue jMonkeyEngine#1903 * Refactored Advanced PBR Terrain to use new for-loops (jMonkeyEngine#1904) * Refactored Advanced PBR Terrain to use new for-loops Similar to my recent pull request doing the same for the base PbrTerrain shader (jMonkeyEngine#1901) this PR also adds for-loop support to the advanced version of the pbr shader that uses texture arrays * Update PBRTerrain.frag * Update AdvancedPBRTerrain.frag * Update AfflictionLib.glsllib (jMonkeyEngine#1905) This PR goes along with my last PR cleaning up the AdvancedPBRTerrain.j3md shader. The method getTriPlanarBlendFromTexArray() was put into this glsllib that contains all of the other commonly used functions for the pbr terrain shaders. * common.gradle: set class files compatible with Java 8 using "release" option (jMonkeyEngine#1907) * common.gradle: set class files compatible with Java 8. This will keep java 8 compatibility when it is compiled with newer java versions. * Merge with existing block. * Reformat code. * common.gradle: add "Created-By" jar manifest to show Java version and vendor name (jMonkeyEngine#1913) * common.gradle: add Created-By in jar manifest to show java version used to create the jar. * Also add vendor name. * ParticleEmitter: improve code readability. Apply the DRY principle (jMonkeyEngine#1912) * add 4 getters to JmeContext for screen position and frame-buffer size (jMonkeyEngine#1911) * test and fix for jMonkeyEngine#1909 (NPE while generating tangents) (jMonkeyEngine#1910) * add a JUnit test for issue 1909 (NPE while generating tangents) * solve issue jMonkeyEngine#1909 (NPE while generating tangents) * jme3-lwjgl: bump to lwjgl 2.9.5 (jMonkeyEngine#1914) * Fix jMonkeyEngine#1917 (RendererException in ScreenshotAppState: Attempting to upload empty buffer) (jMonkeyEngine#1918) * PBRLighting: fix comment describing packed MetallicRoughnessMap (jMonkeyEngine#1921) Updated a comment about the MetallicRoughness map that previously said the Red channel is unused - changed to instead say that the red channel of MR map stores the AO value if AoPackedInMRMap is true. * solve issue jMonkeyEngine#1919 (underflow while generating tangents) (jMonkeyEngine#1920) * jme-core: add a test for issue jMonkeyEngine#1919 (underflow generating tangents) * MikktspaceTangentGenerator: solve jMonkeyEngine#1919 (underflow generating tangents) * main.yml: deploy with jdk17 (jMonkeyEngine#1922) * Fix jMonkeyEngine#1923 (OSSRH artifacts are build with different java version) (jMonkeyEngine#1924) * main.yml: build on pushes to the new v3.6 branch * gradle.properties: next release from "master" branch should be v3.7.0 * resolve issue jMonkeyEngine#1926 (unnecessary dependencies) (jMonkeyEngine#1927) * buildscript: move def of "niftyVersion" to "common.gradle" (shared between projects) * jme3-testdata: rm dependency on "nifty-style-black" (redundant with "jme3-niftygui") * buildscript: mv "nifty-examples" dependency from "jme3-testdata" to "jme3-examples" * solve issue jMonkeyEngine#1928 (OutOfMemoryError in FBX importer) (jMonkeyEngine#1929) * solve issue jMonkeyEngine#1928 (OutOfMemoryError in FBX importer) * tweak the new javadoc * solve issue jMonkeyEngine#1930 (NPE in FbxLayerElement) (jMonkeyEngine#1931) * solve issue jMonkeyEngine#1932 (class cast exceptions in FBX importer) (jMonkeyEngine#1934) * solve issue jMonkeyEngine#1933 (unsupported operation in FbxNode) (jMonkeyEngine#1936) * solve issue jMonkeyEngine#1937 (NPE in FbxObject) (jMonkeyEngine#1938) * solve issue jMonkeyEngine#1939 [NPE in FbxMesh.applyCluster()] (jMonkeyEngine#1940) * FBXCluster: create empty arrays if the cluster contains no keyframes * FbxLoader: don't construct an animation if there are no keyframes * update Groovy to v3.0.15 * Update Application.start javadoc. (jMonkeyEngine#1947) * Update RenderState.setLineWidth javadoc. (jMonkeyEngine#1948) * Fix jMonkeyEngine#1945 (IllegalStateException when running TestAWTPanels with LWJGL 3) (jMonkeyEngine#1949) * solve issue jMonkeyEngine#1879 (compile-time error in Skinning.glsllib) (jMonkeyEngine#1942) * Fix issue jMonkeyEngine#1558 (TestAWTPanels crashes with LWJGL v3 on Linux) (jMonkeyEngine#1944) * TestAwtPanels: apply swing system LAF. This is needed to prevent JVM crash on Linux and LWJGL 3. (See issue jMonkeyEngine#1558) * Use single class imports. * solve issue jMonkeyEngine#1806 (global FrameInterpolator violates threading model) (jMonkeyEngine#1943) * solve issue jMonkeyEngine#1806 (global FrameInterpolator violates threading model) * FrameInterpolator: deprecate the global instance * Fix a typo in LwjglWindow (jMonkeyEngine#1953) * Add WaterFilter.getReflectionView method (jMonkeyEngine#1951) * Added WaterFilter.getReflectionView method. * Update javadoc. * README.md: delete a dead link (Maker's Tale) * README.md: add a link to Wild Magic * resolve issue jMonkeyEngine#1955 (Can not play vorbis audio on Android API 31+) (jMonkeyEngine#1956) * android-native-vorbis: fix double asset file descriptor closure * NativeVorbis: better names and javadocs * NativeVorbisFile#readIntoBuffer: specifies the start and the end of the read * NativeVorbisFile: better explanation for the output buffer on read functions * com_jme3_audio_plugins_NativeVorbisFile.c: refactored logs * NativeVorbisFile: some docs enhances * NativeVorbisLoader: added updated jme3-copyright * [skip ci] update natives snapshot * jme3-jogg: remove dependency on Java Media Framework (jMonkeyEngine#1962) * solve issue jMonkeyEngine#1960 (use jme3-jogg for loading ogg files on android) (jMonkeyEngine#1961) * solve issue jMonkeyEngine#1963 (TestMusicPlayer fails to load AL library on lwjgl2) (jMonkeyEngine#1964) * TestMusicPlayer: fix UnsatisfiedLinkError when using lwjgl2. Solves issue jMonkeyEngine#1963 * Add missing load function for "openal" natives. * Removed the unnecessary check for lwjgl 2 in classpath. * Fix comment * Replace Exception with warning in TerrainPatch (jMonkeyEngine#1966) * Replace Excpetion with warning in TerrainPatch PR following up on the discussion in this thread: https://hub.jmonkeyengine.org/t/terrain-collision-exception/46491/6 * Update TerrainPatch.java * Update TerrainPatch.java * Update TerrainPatch.java * Update TerrainPatch.java * Update TerrainPatch.java * Update TerrainPatch.java * Update TerrainPatch.java * Cleanup NativeLibraryLoader & fix wrong library path (jMonkeyEngine#1967) * Cleanup NativeLibraryLoader * Fix wrong library path * solve issue jMonkeyEngine#1969: missing check in GLRenderer.clearVertexAttribs() (jMonkeyEngine#1970) * solve issue jMonkeyEngine#1975: TestAttachDriver doesn't reset properly (jMonkeyEngine#1976) * Improve NativeLibraryLoader (jMonkeyEngine#1973) * Improve NativeLibraryLoader. * Add javadoc. * Moved library extraction requirement check into a separate method. * Fix javadoc. * Refactor library extraction check method. * Extract natives to system temp directory retrieved by System.getProperty("java.io.tmpdir") instead of working directory. * Renamed enum "Openal" to "OpenAL" and added javadoc on NativeLibraries. * Update comments. * Deploy master branch commits as snapshot (jMonkeyEngine#1983) * Deploy main branch commits as snapshot * Snapshot builds don't have a Release artifact * restrict snapshot to actual commits on master branch * Deploy steps don't actually use test-run maven artifacts * Partial Revert. We need that artifact * Fix sonatype snapshots repo url as mentioned in https://central.sonatype.org/publish/publish-guide/ * Allow use of Emissive color as a multiplier with EmissiveMap in PBRLighting (jMonkeyEngine#1979) * Add support for NormalScale in PBRLighting & GltfLoader (jMonkeyEngine#1980) * Added NormalScale factor in PBRLighting. The scalar parameter applied to each normal vector of the normal map. This value scales the normal vector in X and Y directions using the formula: `scaledNormal = normalize((<sampled normal texture value> * 2.0 - 1.0) * vec3(<normal scale>, <normal scale>, 1.0))`. * Add support for reading NormalScale in GltfLoader. * Add support for AoStrength in PBRLighting & GltfLoader (jMonkeyEngine#1981) * Added AoStrength factor in PBRLighting. A scalar multiplier controlling the amount of occlusion applied. * Add support for reading AoStrength in GltfLoader. * Fix ao calculation to follow gltf specs. * Update comment on AoStrength mentioning the min and max values. * Clamp ao to 0 for negative values that might cause by applying AoStrength > 1. * Use glsl clamp instead of max. * update Groovy to v3.0.16 * Some javadoc cleanup (jMonkeyEngine#1986) * Fix some invalid HTML tags * Correct deprecation annotations * Doc-comments cleanup - Corrected some typos, grammar, etc. - reflowed egregiously long comment lines to comply with column limits * README.md: latest stable version of Engine * LwjglContext: re-initialize renderer on context restart (lwjgl 2) (jMonkeyEngine#1988) * LwjglContext: initialize renderer on context restart (lwjgl 2). * Reset GL objects in renderer when context restart. * README.md: add Mravelous Marbles to the list of JME-powered games * update the LWJGL3 libraries from v3.3.1 to v3.3.2 * update Groovy to v3.0.17 * correctly handle negative IDs in getUniqueId() methods (jMonkeyEngine#1991) * NativeLibraryLoader: more detailed exception in computeNativesHash() (jMonkeyEngine#2001) * README.md: add "Boxer" to the project list * solve issue jMonkeyEngine#2003 (ParticleDepositionHeightMap.load return value) (jMonkeyEngine#2005) * solve issue jMonkeyEngine#2002 (TerrainGridTileLoaderTest fails to load tiles) (jMonkeyEngine#2006) * solve issue jMonkeyEngine#2002 (TerrainGridTileLoaderTest fails to load tiles) * TerrainGridTileLoaderTest: add a clarifying comment * solve issue jMonkeyEngine#2011 (app crashes when using OpenGL version 3.0 and 3.1 with LWJGL 3) (jMonkeyEngine#2009) * fix system crush issue when set desktop AppSetting.setRenderer below 3.2 * Revert "fix system crush issue when set desktop AppSetting.setRenderer below 3.2" This reverts commit 11b7c9e. * fix system crush issue when set desktop AppSetting.setRenderer below 3.2. This fix is reedited by instruction of ali_rs --------- Co-authored-by: ray <raymond.yang@cottonwoodanalytics.com> * solve issue jMonkeyEngine#2007 (instanced objects are culled when using WaterFilter) (jMonkeyEngine#2008) * Fix issue with InstancedGeometry that uses the wrong camera for "instance culling" check. * Minor javadoc update. * Variables should start with lower-case. (jMonkeyEngine#2013) New PullRequest as suggested in jMonkeyEngine#2012 Co-authored-by: Starcommander <starcommander@gmx.at> * solve issue jMonkeyEngine#1992: better messages in spatial assertions (jMonkeyEngine#1993) * jMonkeyEngine#1992 Improve the messages being reported from spatial assertions * jMonkeyEngine#1992 Make clear what name is in the assertion messages * jMonkeyEngine#1992 Whitespace correction * solve issue jMonkeyEngine#2015 (Picture class lacks 2 getters) * CloneableSmartAsset: clarify the setKey() javadoc (jMonkeyEngine#2018) * update Gradle to v7.6.1 --------- Signed-off-by: Kasper Aaquist Johansen <kasperaaquist@gmail.com> Co-authored-by: Scrappers Team <60224159+Scrappers-glitch@users.noreply.github.com> Co-authored-by: Github Actions <actions@users.noreply.github.com> Co-authored-by: Riccardo Balbo <riccardo0blb@gmail.com> Co-authored-by: Scrappers <scrappers.tm@gmail.com> Co-authored-by: Stephen Gold <sgold@sonic.net> Co-authored-by: Florian Frankenberger <f.frankenberger@mobiuscode.de> Co-authored-by: Wyatt Gillette <jefferydeaver2010@gmail.com> Co-authored-by: Lukas-Habring <102620478+Lukas-Habring@users.noreply.github.com> Co-authored-by: Lukas Habring <lukas@Lukas-PC> Co-authored-by: JosiahGoeman <31492985+JosiahGoeman@users.noreply.github.com> Co-authored-by: Paul Speed <pspeed42@users.noreply.github.com> Co-authored-by: Ali-RS <ali_codmw@yahoo.com> Co-authored-by: Toni Helenius <helenius.toni@gmail.com> Co-authored-by: Jan Schäfer <j@nschaefer.net> Co-authored-by: Kasper Aaquist Johansen <kasperaaquist@gmail.com> Co-authored-by: Michael Zuegg <zzuegg@users.noreply.github.com> Co-authored-by: manuelrmo <118840772+manuelrmo@users.noreply.github.com> Co-authored-by: Manuel <Manuel@DESKTOP-6RJH3UF> Co-authored-by: Noeri Huisman <8823461+mrxz@users.noreply.github.com> Co-authored-by: richardTingle <6330028+richardTingle@users.noreply.github.com> Co-authored-by: Ryan McDonough <peanut64646@gmail.com> Co-authored-by: Sailsman63 <lukejsails@gmail.com> Co-authored-by: Raymond Young <gongxi83@163.com> Co-authored-by: ray <raymond.yang@cottonwoodanalytics.com> Co-authored-by: Paul Kashofer <soundmodul@gmx.at> Co-authored-by: Starcommander <starcommander@gmx.at>
Fix #1843
Reported on the forum:
https://hub.jmonkeyengine.org/t/httpziplocator-questions/45859
After comparing the two zips (one generated by www.ezyzip.com and one by linux archive manager both from the same contents and with the same compression level), I noticed that their entry offsets are different in HttpZipLocator.
Following it, I found that the "extraLen" (Central directory (CEN) header extra field length field offset) used here has different values on these two zip files:
jmonkeyengine/jme3-core/src/plugins/java/com/jme3/asset/plugins/HttpZipLocator.java
Line 239 in 5ae543c
in the zip file created by www.ezyzip.com (and also in the JME's "wildhouse.zip" that is on google storage), that value is 0 (which is why HttpZipLocator does not show an error) but in the zip file that @icyboxs uploaded and also I created in the Linux archive manager that value is non-zero (36 bytes).
Removing the "extraLen" fixed the issue.