Mod Manager Improvements

[Delta-473]

I'm trying to centralise improvements/features people want out of mod manager. So if you have feedback put it in the comments so I can add them to the list.

Features

• Build the mod to iso/game files:
  This way we can add support for games where the hooking doesn't work for yet (ex. kh1, bbs)

• Show which mods have updates at startup

• Ability to select git branch for mod (advance feature)

• Research git submodule support for modpacks

Patching commands for yaml

• Platform tag (ex. PS2, PC):
  People can add different assets that are optimised for that platform to the mod.

• Mod version tag:
  Easier to debug issues to see which version people are running

• KH 1 specific file commands

• ReCOM specific file commands

• BBS specific file commands

[Necrofitz]

For awhile I've found the idea of a setting to automatically update mods would be nice. Something that could be tweaked on and where when opened, the Mod Manager would not just detect apps behind on updates, but also download those updates, maybe spit out a confirmation window stating what all mods were updated. Would also be nice if you could mark mods for exemption though, since for example updating the PC version of rando can have several steps to make sure you did everything right, and if the main rando mod, GoA ROM, updates automatically, it could cause more issues than help for people.

I realize this does come with risk (ie: what if a mod is updated by a bad actor and you auto-download a virus), but that risk exists regardless.

[1234567890num]

It be nice if you can add support for multiple BAR subfiles with the exact name & type. Right now it just edits the first one listed, making the other subfiles with the same name and type inaccessible without editing the rest of the BAR file.

Examples:

• Rikysonic/languagepack-en tried to edit ct_p within msn/jp/PO08_MS301_FREE.bar
• KH2FM-Mods-Num/All-in-One tried to edit BOB within map/jp/hb26.map

[Xeeynamo]

I am very happy to see feedback coming in for the Mods Manager. It has been a small side project I created in a few weeks and I did not even know it is used. We will do our best to improve and expand it. I have a few ideas in this regard.

Short term:

• Remove the "OpenKH Mods License". This is useless. The license is empty and we should NOT change the license in mass for hundreds of mods without notice. Every mod should have its own license.
• Mods browser: we should provide a window where you can browse, filter and install mods. The mod feed can be just a txt file on GitHub. This file can be fed by automatically approving PRs from modders who wants their mod to appear in the Mods browser.
• Kingdom Hearts 1 and Birth By Sleep support
• Fully integrate PC support and run KH2FM for PC in the same way the Mods Manager does with PCSX2
• HD textures support

Long term:

• Provide an API to directly interface to the underlying game. It might be a RESTful service that accepts and replies as a JSON. You do for example GET /party/goofy and it will give return a JSON with all the stats of Goofy. A POST /party/goofy will modify it. The server will map the API to the appropriate backend (eg. PCSX2, OpenKH Game Engine, PC Epic Games port, etc.). A scripting language can be built upon that to create mods that works on all versions of the game.
• Provide a way to build a custom version of the game by directly modifying the game files. Topaz already have a working PoC for the EGS version of KH2 BFMM Functionality in Mod Manager. #627 and we can use it to expand it to everything else.

---

• Platform tag (ex. PS2, PC):
  People can add different assets that are optimised for that platform to the mod.

@Delta-473 I have mixed feelings about this. Mods are supposed to run seamlessly regardless the version of the game. If introduced there's the risk of fragmentation. I think it is just a workaround to the main issue. Also could you provide a scenario where tagging the version of the game would be beneficial?

[MainMemory]

That API sounds like an interesting idea, but wouldn't it be a bit slow? Something we could possibly do is create an API that works with the Lua engine that's already available for the PC versions (LuaFrontend/LuaBackend) and in the PCSX2-EX fork of the emulator. It shouldn't be too difficult to adapt that into your game engine, and if we can provide a centralized way to access various pieces of game data, that would take some of the work off of the scripts.

[Xeeynamo]

Yes, we can use LuaBackend as long as it is easy to have proper API that can work seamlessly regardless the version of the game. I never tried LuaBackend but I am a bit confused on what it tries to accomplish: it is to create scripts but where the actual back-end and the API layer is unknown to me.

Regarding the API as a RESTful service it is just my fetish of having a debugger running on a browser from a local or remote device. The same API would be accessible to whatever scripting language anyone wants to choose (LUA, JavaScript, C# and so on). It does not have to be necessarily JSON, it can be also gRPC+ProtoBuf for the best absolute performance. Truth is that I see this as a nice idea but who knows if it will ever come to life.

[Delta-473]

@TopazTK can explain what LuaBackend is the best. ;)

[TopazTK]

About the Platform Tag, this works on an asset basis, so no fragmentation occurs. Quite the opposite. LuaEngine sports something similar to have 1 script support everything.

As for LuaEngine, it is basically a Lua-powered Cheat Engine with optimizations and API made specifically to RAM Edit shit. There is a doc at https://github.com/TopazTK/LuaFrontend/DOCUMENT.md/

[MainMemory]

I think you mean https://github.com/TopazTK/LuaFrontend/blob/mother/DOCUMENT.md
LuaEngine is primarily geared towards real-time memory editing, scripts use it to read and edit things like the save file data, the current room's info, object placements, etc. Right now, each script has to provide the location of each piece of data itself, which varies by platform and can change between versions; it would be possible to have these locations provided by the engine itself through some API, maybe even formatted as a structure that Lua can access directly, although that might be a bit beyond the scope of a project like PCSX2-EX.

[Dee-Ayy]

Scenario for a where a platform tag is useful:
I create a mod that edits the sys.bar.
I want this to work for PCSX2 and the PC port so I create a mod.yml that patches the sys.bar in msg\jp\ and msg\us\.
Assuming the user is using the JP/Default region when using the setup wizard for the PS2 game this patches fine and works as intended for both the PS2 and the PC versions of the game, but it will break if the user is playing on EMU and has their region set to US (for using KH2 English controls such as X to attack).
in that instance the patch to msg\us\ will go through but it'll create a sys.bar that's completely empty except for the edits I wanted to add to it.
This is a major problem because when the Mods Manager region is set to a different region from FM/Default it will prioritize loading the files for that region over the FM/JP files.
This can be resolved by just making multiple versions of the same mod, but with a platform tag the mod.yml can be set up in a way that makes sure both versions of the game can be patched as intended.
Using the scenario above I can set a platform tag for pc that would make it so the msg\us\sys.bar is only patched if Mods Manager was setup using the PC version and that msg\jp\sys.bar is only patched if the setup was done using the ps2 game so that no blank/nearly empty msg files are created due to how binarc works all while conveniently containing everything in a single mod.

Another example is related to how certain files are different between the two versions.
for example MAPs are located in a completely different directory (map\region\ for PS2 and map\ for PC) with a pretty significant difference in that the room plates (the text for the room name that appears when you enter rooms) are inside the MAP for PS2 but for PC are located in map\region\ inside a completely different file type (a BAR instead of MAP).
So if I wanted to create a mod that correctly edits these room name plates completely separate mods would need to be made for both versions of the game, where as if the platform tag exists I could tag specific patch actions to only happen depending on version and contain everything in a single mod.

[shananas]

what about a.fm and a.us files my mods are first written for PCSX so i use a.fm but that is lower quality than a.us and if i want to use the vanilla file from the extraction using type: internal makes the mod manager throw an error on either Emulator or PC then stop building

[Xeeynamo]

I am not currently aware of such issue, but it can be dealt with. One step at the time.

[Dee-Ayy]

So if i'm understanding you right, if I make a mod that edits the jp and en sys.bar to support the PCSX2 and PC versions, these tweaks will make the PCSX2 version load the jp sys.bar no matter the region selected?
If so I feel like this can break certain mods such as the current EN language pack mod rando uses that supports both the FM and US regions for PCSX2.

[TopazTK]

The Mods Manager decides how paths are redirected. Right now it completely ignores the fact that files can be found on different paths for different versions of Kingdom Hearts II. We can do that for PCSX2, if the region is set to Final Mix, then the sys.bar meant for en/ will be used instead of jp/. The same concept can be applied to the maps.

The idea is, rather than assigning the responsibility to the mod to declare its compatibility across PS2 and PC, the responsibility is dropped to the Mods Manager to handle the right paths based on the PS2 or PC version.

Bad idea. This completely breaks compatibility with mods like Dual Audio, which injects files into other regions.

[Dee-Ayy]

I guess to give my final thoughts, I think it's better and easier to add a platform tag for the mod.yml to designate certain patch operation for certain versions of the game. It allows mod creators to specify how exactly a mod should be built for each version. Leaving it to the Mods Manager to decide the paths sounds like a nice idea at first as it could potentially offload a lot of work on creators, but i think it will instead crate scenarios where the creator as to write a far more complex mod.yml to work around how the path redirects work in order to make something that would work for any version as opposed to simply being able to define PC/PS2/Both for operations in the mod.yml.
Also incase it wasn't clear, if a platform tag doesn't exist the Mods Manager can assume the operation is meant for any version of the game if the platform as the additional condition of "any" or if the platform tag does not exist for a specific action.

I'd like you to take these points into consideration and add them as I very much feel the tag would be extremely helpful as I have experience in modding both versions of the game and know the files formats and differences between the two that makes modding the two troublesome at times if I want to make something that will work for both with a single mod rather than creating two version specific ones. the platform tag would alleviate a lot of problems for me and a few others I know of.