ME3Explorer 5.0 Changelog

[Mgamerz]

This is the detailed changelog for ME3Explorer - ME3Tweaks Fork 5.0. It has been 1 year since our release of 4.1, and tons of things changed since. This is a list of highlights as there are way too many changes to describe.

Global new features

ME3ExplorerCore: The core of ME3Explorer, as a library

ME3Explorer has been split in two: ME3Explorer and ME3ExplorerCore.

ME3Explorer

The main UI app you've used over the years, except separated to only contain the editor. It includes things such as the UI and algorithms that make convenience features that wouldn't really make sense in a library that would be used in another project.

ME3ExplorerCore

Contains the majority of the basic backend, such as opening package files and SFARs, parsing properties and binary, reading and writing TLK files, and more. ME3ExplorerCore is a .NET Standard 2.0 library and works on both Windows and Linux (through .NET Core). ME3ExplorerCore does NOT handle texture conversions as that depends on platform specific coders and decoders. You can use this library to power your own projects. Some existing projects that use a 'library' version of ME3Explorer include Mass Effect Randomizer, ALOT Installer and ME3Tweaks Mod Manager.

We have begun to use unit testing against this library to help ensure more consistent code quality. All nightly builds must pass our suite of checks before they're published to ME3Tweaks and available as a nightly build in ME3Tweaks Mod Manager.

Console platform support for READING files

There has been significant work done to support reading console files. They do NOT support saving and as such cannot be modified, but are a useful research asset and good for data mining. Supported configurations include:

• ME3 "Special Edition" on WiiU
• ME1, ME2, ME3 for PS3 (which are all based on the ME3 engine)
• ME1, ME2, ME3 for Xbox 360
• ME3 2011 leak for Xbox 360

Not all features or tools support these platforms, but a majority of the tools do. Notable features that don't work that we know about are:

• WiiU model preview (it uses custom UV compression)
• Textures on most consoles don't work due to them being swizzled or tiled differently, and there's little to no documentation on the formats or how to detile them (in a way that any of the devs can understand)
• PS3 SFAR files can't be decompressed. They list lzma as their compression method, but the actual compressed data doesn't appear to be any known compression type. As such you cannot get a list of filenames for a PS3 SFAR and some entries cannot be extracted.
• Some Xbox files can't be opened, as there's some quirk in LZX compression that trips up the decompressor. This only happens to a few files.

Console file parsing allows us to compare differences between platforms, and at some point may let us port some things from console platforms to PC.

Additional supported unreal versions

The toolset can now open ME2 demo packages, which uses a different version of Unreal Engine than the release of ME2. Strangely, the engine version is NEWER, not older, as the demo came out after game launch. Version 867 of UDK packages can also be opened now, which seem like they were used by 2012-2014 era mods. Not sure why we'd ever need to open them but always good to be able to!

List Dialog click to goto

List Dialogs, which were added in 4.0, provide a list that can be copied. In 5.0, for lists that reference exports/imports, you can double click the list item to go to that item. This is most prevalent in Package Editor.

Not all dialogs support this, but a majority do where it makes sense.

Significant performance improvements

We've profiled several common code areas of the application and identified several areas that could be improved. This includes things such as porting assets across files (We've seen time in this area cut by about 60%), function parsing time (cut by nearly 98%!), and other areas in general.

We've also significantly enhanced our memory leak checks with built-in memory tracking and increased usage of JetBrains dotMemory, which helps us track down memory leaks and tamp down memory usage. We were able to do these through the use of their open source licenses they granted our project. Thanks JetBrains!

Hot Reloading

When AutoTOC-v2.1.asi is installed, saving an ME3 pcc file with ME3Explorer 5.0 while ME3 is running will update the game's in-memory TOC. This allows hot-reloading (by quicksaving and quickloading) for practically any file edit! Without this, any file edit that changed the file size would cause the game to crash because the TOC value would be out of date.

Miscellaneous global changes

• Triple clicking now selects everything in a textbox, rather than just some of the text. Previously it would be stymied by underscores, which are common in things like names.
• Editing ME3 testpatch files works again. It looks like there's a flag in package header that must be set to 1 or game dies immediately on boot
• Reduced dependency on me3tweaks.com
• Entry indexing is now fully done how it is in game - no more 0 based indexing! I'm sure if a developer who doesn't know how unreal internally works see this they'll be like oh no why
• Names are now case-insensitive, but case preserving. This will prevent duplicate same-names being added if there's different casing
• I (Mgamerz) shouldn't have to say this, but we now re-serialize negative zero properly as negative zero. Why is negative zero used in Mass Effect games? I'm not sure. But it's now my favorite number. And yes, negative zero is represented differently from zero, but has the same value. Someday a QA person is going to curse me when they trace the data serialization differently due to it being negative zero, just like I had to

New tools

New tools were added since 4.1 was released. New features for existing tools are listed below this section.

AFC Compactor

AFC Compactor has been fully rewritten and spun out into it's own tool.

AFC Compactor is the AFC version of TFC Compactor, which allows you to 'compact' your AFC file for your mod. As you change audio, there may be orphaned data that is no longer referenced in your AFC file. AFC Compactor builds a new AFC file without this orphaned data, and updates all the audio references. This rewritten version accounts for basegame files which you may have also edited, and can also remove dependencies on official DLC if your mod is only using files from a DLC rather than having a full dependency on that DLC.

SFAR Explorer (replacing SFAR Editor)

Perhaps not the most exciting new tool, but you can use the new SFAR Explorer to do basic browsing of SFAR files. There's no editor functionality, as that's a very antiquated modding style (unless you're modding ME3MP - then you just use ME3Tweaks Mod Manager instead for that feature!)

This version supports opening packages from SFAR directly via right clicking - no disk extraction required.

Live Level Editor

An experimental tool for editing levels while they are open in ME3. Can currently move and rotate NPCs and non-static objects (changes appear instantly in-game). Also has a tool for creating and viewing camera movements in-game.

Animation importer/exporter

Import animations from PSA or UDK files to create new animations or replace existing ones. Export animations to PSA.

Wwise Graph Editor

Advanced audio editing tool for viewing the relationships between HIRC objects in a WwiseBank.

Tool changes

This is an overview of some of the bigger changes the tools have seen in the past year.

Asset Database

Asset Database now includes the ability to create custom filelists which will filter the whole database (each tab) to only show what is contained in those particular files (for example you could add all the files in a particular level or DLC). There have also been various cosmetic improvements.

Package Editor

Package Editor has seen a lot of under the hood changes. There's a lot of small changes, some of which we'll go over here.

Package Editor tabs

Package Editor gained a lot of new useful tools.

ME3Explorer 4.1:

ME3Explorer 5.0:

UnrealScript Editor (ME3 only)
Full decompiler for ME3 Unrealscript. Turns bytecode and script objects into actual Unrealscript.
It can compile function bodies, allowing a much easier code editing experience than hex-editing the bytecode.
Example class decompilation:

Class SFXCameraMode_Explore extends SFXCameraMode;

// Functions
public function Initialize()
{
    local Pawn PawnTarget;
    
    Super.Initialize();
    PawnTarget = GetViewTargetAsPawn();
    if (PawnTarget != None)
    {
        m_pov.Rotation = PawnTarget.Rotation;
    }
}

defaultproperties
{
    Offset = (X = 200.0, Y = 0.0, Z = 40.0)
    HookOffset = (X = 0.0, Y = 0.0, Z = 120.0)
    Begin Object Class=SFXCameraInput Name=s_Input
        CameraSensitivity = (X = 60.0, Y = 60.0)
    End Object
    Input = s_Input
    bIsCameraShakeEnabled = true
}

Movie
Play and replace .bik files (embedded or referenced) directly from within Package Editor or Asset Database.

Location
Easily edit the location, rotation and scale of actors in a StaticMeshCollectionActor or a StaticLightCollectionActor.

Particle module/particle system
These tabs help visualize the exports that are present in a particle system. They don't provide editor functionality (and only work on some parts of particle system), but are helpful when trying to analyze a particle system.

Tree view metadata about exports

Various metadata about exports is now shown, such as tag names, TLK strings, Exec functions, Native indexes, etc.

If you don't like the extra text, you can turn it off in the options.

Multicloning

You can now clone items multiple times at once with the new right click item for Multicloning. You can clone single items, trees, all with your specified porting option.

Porting/relinker changes

• Porting items in that reside under PersistentLevel will result in them being automatically added to the level. Trashing items will also automatically remove them
• Relinker accuracy improved by using the instanced name rather than just the name of the exports being relinked
• Relinker report can now be copied to clipboard
• All objects can be fully relinked (including properties, binary, and function bytecode)

Miscellaneous changes

• You can now create a new, blank ME3 level file from the File menu
• Compare to unmodded now leverages the ME3Tweaks backups that are created from ALOT Installer, Mass Effect Randomizer, and ALOT Installer. When comparing, these files are listed first as they reside in a backup are more likely to not be modified
• Compare to unmodded now supports reading from SFAR files. You can now compare packages you receive to unmodded without having to have your DLC unpacked
• A new debugging menu item for finding invalid object references in object properties has been added
• The 'busy' indicator now is fully drawn above the hexbox, rather than below it if the hexbox is visible. The hexbox is hosted in Winforms, since there's no decent WPF hexbox options. Due to the interop they use the hexbox would appear on top of things drawn above it
• The super annoying bug where Package Editor would randomly skip back to what was previously selected has been fixed. I know this is the one everyone cared about finding in this changelog
• You can now go backwards and forwards in the tree using the back and forward buttons on your mouse, if you have those buttons
• You can now search backwards by holding shift and pressing enter in the search box and class search boxes
• Searching now includes instance, so you can do like SearchForMe_2 to find the 2nd instance
• More information in package header info display

Pathfinding Editor

• You can now make and break spline connections
• The options for what to view has been reorganized into Art/Design categories
• There are now multiple more visible types in pathfinding editor, such as DecalActors
• Crazy experimental tools have been added, such as the ability to recook a level which removes all unused data.
• New basic experiments, such as shifting levels by X Y and Z as a whole
• You can now clone collection components directly
• Added brute to list of sizes for a node/path

Dialogue Editor

Minor improvements to performance and cloning.

Interp Editor

• Now shows tooltips that show the values of the track at that point on the timeline
• Now supports ME1, ME2

Sequence Editor

• Sequence Object Toolbox now works with ME1 sequences.
• Displays progress bar when performing long-running operations instead of freezing.
• Added "Search by Comment" feature, especially useful for finding things in dialogue sequences.
• Added a 'tight loop' detector, which can find items which directly input back into themselves (that are not a gate). These severely harm game performance and should not be used unless by a gate.
  
• Add Originator to SFXSeqDeath, to make it easier to figure out what the source trigger actually was
• Fix opening sequences contain sequencereference objects that are imports (ME1 issue)

Meshplorer

• ME2/ME3 mesh importing has a workaround for the crazy animations that occur when importing: Import single LOD. Something messes up the file when exporting/importing from UDK, we aren't sure if it's UDK or UModel. Single LOD seems to fix this
• Now has filtering for types that can be shown
  
• Throw warning to user if the origin rotation changes between pre and post import

AnimViewer

Browse ME3 animations, and view humanoid ones on your squadmates.

Plot Manager

• Descriptions are shown on the Task tab of quests, where there associated StringRefs. Substates are handled properly in state event editor

Package Dumper

• Now dumps functions again, now that we have reliable function parsing
• Fix exception that could occur if multiple same-named files were dumping at the same time. Now they will include their source name, such as Basegame or DLC name

Soundplorer

Soundplorer has received a few useful updates for developers.

• TLK strings shown for export names that contain TLK references
  
• Can now parse Big Endian AFCs and ISBs. These use a different codec and magic number
• Much faster sound length calculation. We no longer parse out the audio to determine the sound length - we can compute it's length using the sample count, channel count, and the sample rate. This means parsing out the file is much faster and doesn't require writing to disk
• There is now a link to download Wwise that is required for authoring ME3 audio on the startup page for soundplorer, along with a new recents menu

Mount Editor

• Added additional mounts that exist in ME2 files. We don't fully understand ME2 mounts
• Save dialog now defaults to the existing loaded directory, if available. Less annoying to save a mount file

Export loaders updates

Export loaders are items that are re-used around the toolset. You likely know them as the tabs in Package Editor, but you have seen them shared elsewhere, like Soundpanel in Soundplorer and the mesh renderer you see in Meshplorer.

Interpreter

Used in multiple tools

• More IntProperties that refer to a TLK string now show their TLK string value
• The Add Property dialog now supports searching
  

Mesh Renderer

Used in Package Editor, Meshplorer, Asset DB

• More accurate ME1 texture lookups when trying to determine the diffuse texture to use
• Fix rendering of things where material's link is 0 (root), which is common in ME1 packages
• Optimize how assets are loaded by ensuring already-loaded imports are not attempted to be loaded again
• Optimize by not loading an additional copy of the same texture if the item is referenced multiple times
• Fix mesh renderer not working if launched from a file association due to working directory being different
• There is now camera information
• Rendering no longer occurs if the tab (in Package Editor) is not open
• You can now render Models and ModelComponents
  

Binary Interpreter

Used in Package Editor, Meshplorer, other tools

• Improvements to multiple scans

Soundpanel

Used in Package Editor, Soundplorer, AssetDB

• Can now play ME1 audio located in SoundNodeWave. We can now play all ME1 audio! That means we have full audio playing for the trilogy now
• We now use vgmstream to convert audio

JPEX SWF/Scaleform launcher

• Now exports additional assets that JPEX will often ask for. Not sure how useful this is, but maybe it'll improve how JPEX behaves

Bytecode Editor

Used in Package Editor

• Should now be able to parse all games functions and states on all platforms
• Fix some natives not showing their names in the token list
• Cursor position in the hexbox now shows the opcode in the status bar. This won't work for extended natives (which are 2 byte opcodes)
• MEMORY OFFSETS ARE NOW SHOWN IN ME3 (+PS3 ME1/ME2). This means you can MUCH more easily edit bytecode by hand as you can now tell where if and goto's actually go!

Metadata Editor

• Prevent setting link to self. An exception will be thrown instead