.pk3 Files

[UA106]

Is it possible to use the .pk3 files from any Quake III engine game? I wanted to try my Jedi Academy .pk3 files on here.

[stevezau]

I Also want to know how to load my own maps? I have my local server running, added the pk to baseq3 folder, i can see it detects it but says not in manifest.
when i try change to map i get map not found.. any idea where i can t-shoot?

[begleysm]

I haven't, personally, done anything with custom maps. However I do have some steps on how to do this from a guy I was communicating with over email (regarding a Windows setup). I've had it in the back of my mind to walk through them and add a section to my How Tos but I haven't gotten around to it yet.

Here are the relevant excerpts from my email conversation with him:

Ok so I managed to load custom maps. The method I used is basically the same as the one to upgrade demo to full version.
This is what I did to keep pak0 to minimal.
First a took pak0 from full version and extracted just the textures folder and put the textures folder inside demo pack0.
Then I got two maps I like and put them inside demo pak0 as well with all their levelshots, bsp, aas, and so on.
And then I basically just did the method of identifying the numbers for the pak0, and replace them in the 3 js files. Then a copy of pak0 to game server base and number-pak0.pk3 to base of www folder. Also editing the manifest with proper size in kb and number.
So the method is a pain in the ass but at least custom maps are possible to add and you don't need to have the huge full version pk0. I added a couple of cool maps and mine is just under 90 mb.

Ok finally i made a great pak0.pk3.. with my favourite competitive maps.
As i said before the method is the same as for the full version transformation.
Its all about customizing a pak0.pk3 and leaving the rest of the paks the same.
I ended up taking the original full version pak0 and deleted all maps, all player models, video and music.
Just placed the player models i liked most, along with their respective bot files and scripts and the maps i personally love.
Before doing this i tested everything in my regular q3 installation and everything worked fine.
Then i did the editing of JS files, manifest, copy the pak to gameserver and www. etc.
It works spectacular !

The map is one extracted from QuakeLive and this is my map cycle.
set d1 "map qzwarmup ; set nextmap vstr d2"
set d2 "map acid3dm6 ; set nextmap vstr d3"
set d3 "map acid3dm9 ; set nextmap vstr d4"
set d4 "map mptourney4 ; set nextmap vstr d5"
set d5 "map pukka3tourney2 ; set nextmap vstr d6"
set d6 "map q3tourney5 ; set nextmap vstr d7"
set d7 "map acid3dm2 ; set nextmap vstr d1"
vstr d1

I'm not sure if he has a handle on github. In order to give him a bit of credit, here's a link to his webpage
http://www.grabisoftnet.com

[ghost]

I really like this idea of splitting up the menu system and maps and uncommon textures in to their own pk3s. Thank you inolen!

I have had just over a few people reach out to me personally regarding this. We wanted to run Defrag mod. When we tried to use repack.js, I ran into errors with some of the naming conventions used inside some shader files. Repack, and it's dependency, quakejs-files wasn't very helpful, and the process failed to complete. I'd really like to attack this workflow this weekend. Here's my approach:

1. Use the ioq3-ded.js, the dedicated server build, as a command line tool. This would coincide with the commands available in sv_ccmds.c, I think there is even already a list_assets type function for debugging maps. https://github.com/ioquake/ioq3/blob/master/code/server/sv_ccmds.c#L1543

2. Insert the manifest.json/repack.js logic in the server, no matter if it is compiled as native or emscripten, the server can generate a manifest.json of loaded assets on demand.

3. Support 302 MOVED redirect on the server. I am not sure how this works. If an admin allows downloads, it is downloading over the UDP connection? Anyways, it would be cool to add HTTP support to both client and server just for the purpose of supporting a CDN like setup. We can all add our pk3 files to a bucket on AWS or GSC and use Cloudflare CDN to distribute it to anyone who wants to play. Adding HTTP support would probably be somewhere around here: https://github.com/ioquake/ioq3/blob/master/code/server/sv_ccmds.c#L1543

4. Rejoin the interface with the little HTML loading bar from quakejs, remove the requirement for the demo linux image (this is a redundant download even on current quakejs) and all other pk3 files, just to get the menu up and running as fast as possible. Remove the loading bar and use the standard loading screen built into Quake 3. Here:
   https://github.com/ioquake/ioq3/blob/master/code/ui/ui_main.c#L5463

5. These changes wouldn't restrict me from using cloud functions or AWS elastic transcoder in my content pipeline.

[ghost]

I am currently in the process of removing this restriction where cUrl has been disabled and server downloads are disabled. This should make customs pk3s a more traditional experience.

https://github.com/inolen/ioq3/blob/4f7d7bf2159aa0a18b79bb417aa760abac817b2a/code/client/cl_main.c#L2501

This is excluded from QuakeJS because it's assumed you don't want 300MB laying around in your browser storage. With my new changes, the server can tell the client which pk3s to download for custom maps. Instead of using cUrl it will use XHR with progress updates! :D

[stevezau]

@briancullinan nice, is there somewhere i can watch or get updated when your changes are merged/released? Is there a PR?

[ghost]

Wow, this has been an incredible journey. I have found some weird, weird stuff. q3ctf5.aas doesn't have a BSP map file. So q3ctf5 doesn't exist. This led me to the website https://tcrf.net/Quake_III_Arena The Cutting Room Floor that investigates these kinds of missing files and assets and speculates on their original purpose!

There are about 200 files in baseq3 that are never used. No sense in forcing players to download those! Including icons/iconf_blu only icons/iconf_blu1-3 are ever referenced. I now know more about this than I ever wanted to!

Using the --edges parameter will allow the repack script to duplicate certain files, for example, if 2 maps share 4 textures, it may make sense to make the user download those 4 textures for a total of 8 files, rather than group them with a bunch of shared textures and have to download 8 files you don't need/textures apart of maps not in rotation, for a total of 12 textures.

I will tidy this up with some percentage outputs, it should be much more generalized, fault tolerant, and also more specific. Running on baseq3 created ~160 paks that can also be fine tuned using the edges parameter.

A sample of repack in info mode:

Known files: 1041/3777 - 27.6%
Known directories: 3666/3777 - 97.1%
Menu files: 877/3777 - 5810202 bytes
Cgame files: 1142/3777 - 30.2%
Not found: 623
Files in baseq3: 1
Most used assets: [
  '209 - /users/briancullinan/planet_quake_data/baseq3-combined-converted/sound/world/fire1.opus',
  '75 - /users/briancullinan/planet_quake_data/baseq3-combined-converted/sound/world/firesoft.opus',
  '43 - /users/briancullinan/planet_quake_data/baseq3-combined-converted/sound/world/wind1.opus',
  '42 - textures/common/nodrawnonsolid',
  '36 - /users/briancullinan/planet_quake_data/baseq3-combined-converted/textures/base_light/ceil1_39.blend.jpg',
  '35 - textures/common/clip',
  '35 - textures/common/caulk',
  '34 - textures/gothic_floor/metalbridge06brokeb',
  '33 - flareShader',
  '32 - textures/common/trigger'
]
Shared files: 654/3777 - 17.3%

[stevezau]

Hey, @briancullinan I plan to look at adding custom maps this week. I read the above but i am not sure i'm following.

It seems you are digging into the code, making modifications to make the whole process simpler? Is that right? It is a WIP Right?

[ghost]

The repack script I wrote has too many missing files, I think I will only use the conversion feature, and skip the whole graphing and repacking concept. I will have to modify the renderer and probably add this:
https://github.com/Chomenor/ioq3_new_filesystem

[stevezau]

ok, not sure i am clear what instructions to follow but ill give it a go today.

[XanderStrike]

I'm kind of baffled by your posts @briancullinan, it sounds like it took reimplementing parts of ioquake? I don't really know where to begin if I wanted to utilize your work. From a user's perspective quake.games is incredible though, the speed the levels load is amazing and it looks like most maps work including the custom ones you've added. Very cool. If you're up for it, I'd love a start to finish setup guide.

I've also been trying to do this with a a more straightforward approach using the tools in this repo and have had some success. Based on my reading of the docs I would imagine that all you need to do to get custom maps working is to get your content server to serve them, and the way to do that is add them to the source asset directory used with repak.js. That said, I'm not a quake or javascript expert.

So far I've successfully repak't and run the asset server against the full game plus some custom maps, and have successfully been able to launch all the maps using the /map command. It took a small patch that I've opened a PR to @begleysm's fork to get repak to run, but it seems to be working well.

However I haven't been able to host a dedicated server with any maps besides those included in the demo, which sorta defeats the purpose. I think the dedicated server is downloading the demo somewhere and the maplist comes from that, haven't figured it out yet but I'll keep looking later this week.