Some enhancements

[revelator]

added the darkmods AVX and AVX2 simd instructions.
hybrid ARB2 / GLSL renderer the GLSL part only handles interactions eg. world and can be disabled completely without any bugs in case someone wants to use say sikkmods pom shader.
RAII based depthbounds testing code.
optimized VBO code -> originally from Michael Hiney (InsideQC).

Had some problems with SDL2's cpu feature detection so i added a modified version of Darkmods.
Problem being that SDL2's Cpu detection code does not seem to detect anything above SSE3 (i dont know much about SDL but the code i replaced also works with gcc/clang).

runs pretty fine here and i was asked by one of the TDM devs about a review.

the GLSL part is not strictly nessesary but add's a little for modders who prefer writing shaders for it.
the GLSL part does not handle shadows nor material / user shaders but it could easily be modded to handle shadows as well.
handling material / user shaders would be a good bit harder though.

here is a compiled version for windows for testing.
dhewm3_2.zip

[DanielGibson]

Interesting.
Could you link the source?

[revelator]

dhewm3-git_2.zip

here you go :) i used SDL2 and other libraries from vcpkg for my build.

[revelator]

ok after checking a few things i got SDL's cpu detection working.
one minor gripe SDL2 does not have a check for FMA3 which is required for AVX2 but i have to suspect the SDL devs know they're stuff so im just passing the CPUID_FMA3 flag if the detection says AVX2 is supported.
For SDL1.2 i keept the detection routine for SSE3 AVX and AVX2 since it only supports up to SSE2 (i use a check for SDL's VERSION_MAJOR to check for SDL above version 1).

I forgot to add in a small codepiece from darkmods AVX detection which takes care of an improper _xgetbv intrinsic in gcc < 9.0 (only used if SDL is 1.2).

mostly cosmetic also added a check for CPU type so that it now prints AMD or Intel instead of generic which it now only prints if forced or if detection fails (disables all cpu intrinsics).

[DanielGibson]

dhewm3-git_2.zip

here you go :) i used SDL2 and other libraries from vcpkg for my build.

Thanks - do you have a git repo with the source?
Where the changes are ideally split up into commits?
If you create a pull request the code changes could be discussed there (and you could easily share additional changes by just pushing them to the branch used for the PR)

[revelator]

Been on hiatus for some years just decided i wanted to get some changes out that might benefit your port.
No github repo im afraid but i could make one though it might take a little as im in the process of having dentures made for both my upper and lower jav.

Actually also managed to port grimms quest for the gatherers key as i have permission to use his game sources.

[DanielGibson]

There's no hurry, good luck with your teeth! :)

Actually also managed to port grimms quest for the gatherers key as i have permission to use his game sources.

Cool, if you also have permission to release the source under GPL, that would be a great addition for the dhewm3 mods!

[revelator]

That should be ok, as far as i understood him when i last talked with him the restriction was more towards the materials he created for the game (textures models etc) not so much the game code itself which is in large parts based on sikkmod.

was actually rather nasty to get the code ported hehe :)

[revelator]

grimm_2.zip

heres the compiled mod dll for it if you want to try it out.
softparticles and the shader based gamma should be turned of as it looks wrong with it (way to dark).
it sadly overuses the DOF feature from sikkmod way to much so it looks kinda blurry in places.

[walikor]

Am I wrong or is this the required source code? I found it by downloading the mod on moddb
grimm_source.zip

grimm_2.zip

heres the compiled mod dll for it if you want to try it out. softparticles and the shader based gamma should be turned of as it looks wrong with it (way to dark). it sadly overuses the DOF feature from sikkmod way to much so it looks kinda blurry in places.

[revelator]

yep thats the source allright :)

[revelator]

well i allready ported it so unless you want to try it yourself i could just upload mine ?.
if you want to try porting it yourself it is easiest using your sikkmod port as a base.
there are a few things it does differently than sikkmod and also a rather glaring code whoopsie where it does a binary comparison of a string x-D

[DanielGibson]

I'd prefer not to port it myself if the work is already done - the source would be welcome, even as a zip (in case of mod ports I don't care that much about commits) :)

[revelator]

dhewm3-sdk-sikkmod_2.zip

ok here you go :)

[revelator]

doh forget to rename the folder before zipping it up... well as you can guess i used the sikkmod port as a base for the port. just rename it to grimm :)

[revelator]

also fixed a lot of the warnings but a few remain so there is definatly room for improvement.

mostly compiler complaining about unreachable code.

[DanielGibson]

thanks! :)

[revelator]

dhewm3-sdk-grimm_2.zip

last minute fix seems it did not compile for x64 works now.
also renamed the folder :)

[revelator]

ah well the game starts but crashes to desktop when starting a map in x64 so definatly something fubar still.

[DanielGibson]

No worries - should I wait some more (days? weeks? don't matter, there's no hurry) or should I try to port the source myself?

[walikor]

well i allready ported it so unless you want to try it yourself i could just upload mine ?. if you want to try porting it yourself it is easiest using your sikkmod port as a base. there are a few things it does differently than sikkmod and also a rather glaring code whoopsie where it does a binary comparison of a string x-D

thanks now the grimm mod works, the only thing missing is dungeondoom and I think dhewm3 is compatible with all mods

[revelator]

well it runs fine on x86 so must be me missing out on some pointer changes for x64.
as you probably know the code better it should be fixable without porting everything again.

[DanielGibson]

Ok, I'll look into it :)
I indeed have some experience with chasing down 64bit issues in this code

[revelator]

not a bug as such rather a missing function, my hybrid GLSL renderer chokes on the shader based gamma making things way to dark if enabled turns out that even though it is loaded as a material shader it also needs some some code for it in the GLSL interaction routine... so for now i disable shader based gamma if using the GLSL interaction renderer. i also added it to imgui and defaulting it to off for now.
softparticles work even with GLSL on as it does not need extra code in the interaction renderer :)

[DanielGibson]

https://github.com/dhewm/dhewm3-sdk/tree/grimm works on 64bit Linux, didn't test Windows.

Sorry, but you did something wrong when porting: All the GPL license headers were replaced with the SDK license headers - probably you just copied the Grimm source over the dhewm3 SDK?
That would also explain why it didn't work on x64, it was probably missing 64bit fixes that dhewm3-sdk has.
I did a fresh port based on the the original Grimm source and it seems to work.

The best way to port a mod to dhewm3 is described here: https://github.com/dhewm/dhewm3-sdk#how-to-port-a-mod-to-dhewm3

[revelator]

Heh yeah as said you probably know the source better :) actually i went the other way around at first i hand edited all the grim source files picking includes as they were in the sikkmod port and looking for anything commented with -DG after that i did indeed copy them over x-D

[revelator]

works in windows to, also the 64 bit version :)

[revelator]

for those not in the know, shader gamma breaks certain sikkmod features like SSAO (well newer worked to well to begin with) but other parts also show some breakage to a lesser degree.

these settings in my autoexec.cfg lets it look the best ->
seta r_useGLSL "1" /* Only applies to my branch for normal dhewm3 just remove it /
seta r_useDepthOfField "0" / Looks ghastly /
seta r_useMotionBlur "0" / NO! /
seta r_gammaInShader "0" / Breaks sikkmod features */

if you have a modern card it should be safe to set ultra.
SSAO looks great when it works but has a tendency of showing outlines on models, this is due to doom3 not originally having access to the depthbuffer so he had to make some hacks to get it working and they are not compatible with real depthbuffer access.
DOF looks horrible tbh, not sure if it also relates to the depthbuffer stuff but i assure you you will notice :)
motion blur does not work without DOF anyway.

[revelator]

sys-avx.patch
idlib-avx.patch

the changes for sse avx and avx2 plane culling from the darkmod.
next up the changes to use these features.

[DanielGibson]

If you could create a pullrequest I could review the changes there, and comment directly on the corresponding lines

[revelator]

when i get a fork set up :) was more to let you see some of the changes (some are rather large, the shadow code gets a pretty big rewrite for one).

[revelator]

i could actually also add darkmods soft shadows if wanted (not shadowmaps this was added later and requires a lot more code) but im not sure if it will break any mod related soft shadow code like sikkmods which can be a little buggy at times (also pretty slow).
darkmods also takes a performance hit but not to the same extent.
let me know if wanted ?.

[revelator]

AVX* changes added as pull request.

[revelator]

ok i finally got it building on ubuntu (used the wsl version for testing).
im going to make the current fork a draft to test for build problems so lets not commit anything yet ill retire the fork when everything works and start adding in only the changes nessesary to a new one.

as im not fluent in git feel free to chime in with advise on how to use git gui (im using tortoisegit on windows not sure if that can also filter whitespace changes ?).

also builds using mingw-w64 and clang on windows now.
ubuntu build crashes in wsl2 when run with an assertion in imgui but im not sure wsl2 is the best testing environment ?.
clang and mingw-w64 builds work on windows but due to it using the packages from msys2 the game ends up with quite a load of dependencies (curl being the worst).

[DanielGibson]

as im not fluent in git feel free to chime in with advise on how to use git gui (im using tortoisegit on windows

I'm not very familiar with tortoise git. Does it also install the "git bash" shell and the usual git tools? Or just its own thing?
I have the regular Windows version of git from https://git-scm.com/downloads/win installed, and my context menu looks like this:

So I can select "Git GUI Here" to open git gui for the project of the current folder.

If you don't have this, install it. Either from the link above, or, as suggested by the Tortoise Git Documentation from https://gitforwindows.org/ - in the end both lead to the same downloads.
The installers for Git for Windows contain Git GUI, though it could be that they only optionally install Git GUI (and the related gitk tool) and that the Shell integration (Explorer Context menu entries for "Git Bash Here" and "Git GUI Here") is optional as well; in that case make sure all that is enabled in the installer.

Git GUI overview

(Note: This and the following screenshots are from Linux, but Git GUI looks and behaves the same. Also note that usually the diff view uses red and green for removed and added lines, I patched mine to use red and blue because I'm red-green colorblind.)
I have added those big red numbers, they're not actually there.

1 Unstaged Changes

This is the list of files with Unstaged Changes. Those are changes that have been made but are not part of any commit (not even the one you're currently creating).
Adding changes to the commit you're currently working on is called "staging changes".

• The little blue file icons left of the filenames indicate files that are already known to git
• The white (blank) file icons are for new files that are not yet handled by git at all.
• Little file icons with a question mark indicate files that have been in git (up to the last commit) but are now missing, i.e. have been deleted (I deleted AF.* for demonstration purposes).

If you click one of those file icons in the Unstaged Changes list, all changes in the file will be staged (or in case of removed files, the fact that it's removed is staged, or for new files it gets added to git).

You can also select multiple files from the list (as usual, with click+Ctrl or click+Shift) and then click the Commit menu and there Stage To Commit (or press Ctrl-T).

2 Staged Changes

This is the list of files with Staged Changes. Those are changes that will be part of the commit you're currently creating (unless you unstage them again).

• File icons with a green checkmark indicate files that have changes
• File icons with a red x indicate files that are removed (i.e. you staged a file with a questionmark icon in the Unstaged Changes list - without that the fact that the file is removed would not be committed!)
• A white/blank file icon indicates new files that are being added to git with the currently edited commit

If you click one of the file icons in the Staged Changes list, all changes in the file will be unstaged, which means it will just turn up in the "Unstaged Changes" list again.

Again, you can select multiple files, click the Commit menu and this time Unstage From Commit (or press Ctrl-U) to unstage multiple files at once.

3 Diff View

The diff view shows the changes for the currently selected file.
You can select a file by clicking the filename/path in the Unstaged Changes list or the Staged Changes list.
If you selected a file from the Unstaged Changes list, the diff view shows all unstaged changes in that file, i.e. the changes that exist in your working copy, but haven't been committed yet. The diff view also allows to stage just parts of the file; more on that below.
If you selected a file from the Staged Changes list, the diff view shows all staged changes in that file, i.e. the changes that will be part of the currently edited commit (unless you unstage them again). Again, the diff view allows unstaging parts of the file, see below.

4 Commit Message

This allows you to write the commit message for the currently edited commit.
You should leave an empty line after the first line.
You should also generally try to avoid lines that are longer than that input field, unless
you can absolutely not avoid it (e.g. because you want to add an URL to the commit message and it's too long).
When the commit is done, you can click the [Commit] button to the left of that field to create a commit with the message you entered and all changes that are in the Staged Changes list.

• The [Push] button allows pushing local commits to a git server.
• The [ ] Amend Last Commit checkbox allows editing the last commit, for example if you have already clicked "Commit" but noticed that you were missing changes or made a typo in the commit message or whatever. If you have already pushed the commit, amending it will lead to conflicts! In feature branches for Pull Requests this is ok, you can just force-push the changes, but don't do this in the main/master branch or similar branches that other people use!
• The [Rescan] button makes Git GUI check if there have been any changes since Git GUI has been opened, for example because you edited source files in your code editor.
• The [Stage Changed] button stages all Unstaged Changes - usually you should not do this, but instead stage changes using the diff view. Remember that even if you want to stage all the changes of several files (or add several new files), the Unstaged Changes list lets you select multiple files that you can stage with Commit menu -> Stage To Commit (or by pressing Ctrl-T), which IMHO is a better way to do this with more control.
  OTOH you can of course still unstage changes to clean up when using this button.
• The [Sign Off] button adds a line to the commit message that's like Signed-off-by: Your Name <your@email.example>. It's mostly relevant for the Linux kernel workflow and similar, not for most projects on Github or similar code forges.

Diff View details

You can right-click in the diff view to get a context menu:

This is how you can commit (actually: stage or unstage) just some changes.
Stage Hunk For Commit would stage all changes from the Hunk you right-clicked in.
A "Hunk" is basically a block of changes that are close to each other - after one line that starts with @@ -123,45 ..., to the next similar line (or end of the file if it was the last or only hunk in that file).
Stage Line For Commit would stage just the line the cursor is currently in. So in this example, the removal of the headModel = spawnArgs.GetString( "def_head", "" ); line would be staged, but the added lines below not.

Of course (un)staging just a single line isn't always that much fun, so Git GUI also allows you to select multiple lines:

(Note that this time I chose a file from the Staged Changes list, so it says Unstage Hunk From Commit. Of course selecting multiple lines works for both staged and unstaged changes.)

So now you can (un)stage all selected lines at once.
Note that only whole lines can be committed, so even though in my screenshot I've been a bit sloppy and the first and last line are only selected partially, Unstage Lines from Commit would unstage the full lines.

---

I hope this helps getting into Git GUI, and that it's clear now how to commit just relevant changes (scroll through the diff view and stage the hunks you want, or select the lines you want and stage them - or first stage the whole file and then use the diff view from the Staged Changes list to unstage changes you don't want in this. Both ways are valid and useful, depending on the situation).

By the way, if you're done committing changes and have Unstaged Changes that you want to get rid of, you can select the file(s) in the Unstaged Changes list, click the Commit menu and select Revert Changes. Afterwards the selected file(s) will be in the state of the latest commit (probably the one you just created) and the other local changes to them are gone.

If anything is unclear, please ask!

[DanielGibson]

(I should probably turn that into a blog post)

[revelator]

thanks for the excellent explanation, tortoisegit is a little different and im not sure you can select fragments like in git gui.
it also does not install the official git but depends on it so i do have it (time to get aquainted with the built in gui).