It’s been an eventful couple of months, so there are several things I’d like to talk about today.
New version of the SA1 analog fix
You may have heard of the analog deadzone fix for SA2 DC by OnVar. I was looking at input code in SA1 and found many similarities, so I made a request to woofmute, who successfully ported OnVar’s fix to SA1. The fix is available for the US version (1.005) and the Autodemo. I’m not sure whether the issue is addressed completely or if the fix has any side effects, but it appears to work and it’s definitely better than my old hack, so I suggest you give it a try.
The fix injects custom code into 1ST_READ.BIN. The patched bytes for the US version are as follows. With some patience you could turn it into a very long CodeBreaker code…
;Input function hijack
411e2=0ce0f6f3f5f34c8925c708f235f2488908a10ce0
;Input Patch block 1
41284=20e0f6f304c708f2f5f3b28935f2b08920e0b6aff7ff
;Input Patch Orig code reposition
41406=f7ff20e0f4aef6f3
;float for block 1
4129c=000040c5
There are two ways to get the patch:
If you have the TOSEC GDI of SA1 US 1.005, you can apply an xdelta patch to track03.bin. It works the same way as the SA2 patch in the description of the video linked at the top of this post, so you can use the video for reference.
To apply this fix and more, you can use the new Dreamcast Image Builder that I will talk about in the next section. The Autodemo version of the fix is also available for Dreamcast Image Builder.
Release of Dreamcast Image Builder
This is an update to SA1-DC-HD, a project that originally started as a helper tool to add various enhancements to SA1 DC for playing on the flycast emulator. The project has grown to become a universal patcher for Dreamcast games.
As you can see, it’s heavily inspired by the SADX Mod Loader/Mod Manager and uses the same concept of “mods” to add hacks to games. Of course it’s a lot more limited than the actual Mod Loader, but you can do neat things with it. For example, to test its capabilities I made a mod that replaces Deer with Sheep from SA2:
(Chao body parts not implemented yet)
You can check it out here. All mods from SA1-DC-HD are built-in, so you can use it right away. The wiki explains how to use the tool and develop mods for it.
New Autodemo research
You may have seen new videos uploaded by Speeps and woofmute which demonstrate various debug features of the Autodemo and levels that didn’t work before, such as Casinopolis. This was made possible thanks to the research done by Speeps, woofmute and other members of x-hax, as well as previous research by evilhamwizard, ItsEasyActually, supercoolsonic and others.
As you may know, parts of the Autodemo come from different builds of the game that were at different stages of development. When the Autodemo was put together, only the levels that were meant to be shown in the demo were recompiled, and the other levels were left over from older builds (for most of these, textures and lighting files were also missing). We can tell which binaries were recompiled by looking at file dates on the Autodemo GD-ROM:
You can see that there are three “sets” of files sorted by last modified date. The files that don’t boot were compiled against an earlier version of the game’s main binary (1ST_READ.BIN), so the pointers to functions and data in the main binary are all invalid in these files. However, it’s possible to find the functions and data they were meant to point to by comparing the pointers to known functions and data between these files and the files that boot.
Earlier work on this, which demonstrated partially functional Windy Valley and Sky Deck, was posted by evilhamwizard on Sonic Retro. More recently this was picked up by Speeps, who started putting together a list of known broken pointers and their correct counterparts. As the pointer list grew, more levels became functional when known pointers were replaced. To facilitate mass pointer replacement and game testing, I added Autodemo support to my SA1-DC-HD project (now Dreamcast Image Builder). As we were experimenting with the Autodemo, I adjusted the image builder’s mods system to be more flexible and easier to use and converted (together with woofmute) Speeps’ modifications into mods for it. I also repurposed some of my previous work to find missing textures by matching the names in the Autodemo’s texture lists against textures in the final version, the E3 build, the Autodemo Windy Valley mod for the PC version (where missing textures were recreated by its developers) and Dreamcast SDKs (which surprisingly have some early SA1 textures).
As a result of this combined effort, we got the game to show stuff that was never seen before. For example, here’s a preview of the old version of Chao Race restored by Exant:
Or a very early version of the Mystic Ruins:
Or woofmute’s restoration of the debug object placement editor:
Or a super early version of Twinkle Circuit:
Of course, you also have Speeps’ near-complete restorations of Twinkle Park, Lost World and Sky Deck. The biggest restoration so far is the early Windy Valley, the holy grail of the Autodemo for many SA1 fans. It’s based on Speeps’ and evilhamwizard’s pointer fixes and uses textures from the 2018 PC mod by supercoolsonic & ItsEasyActually.
There’s a lot more stuff released and in progress. We now have a public repository with some Autodemo mods, which you can apply with Dreamcast Image Builder to get the restored functionality and content ingame.
Note: Not all of the mods work on hardware at the moment. flycast is recommended.
TLDR version: There was a Korean version of SADX PC released back in 2004. I ported the Korean translation to the Mod Loader, so you can now play SADX in Korean if you install this mod. The mod is compatible with Dreamcast Conversion and Dreamcast DLC but incompatible with HD GUI.
Four years ago I got my hands on a somewhat rare version of SADX PC. I was completely unaware of its existence until I accidentally ran into a listing on an auction, so I got it right away. I’ve mentioned it before in a previous post, and today’s mod release brings the occasion to finally talk about it again.
Introduction and game package
This version was released in July 2004 in Korea by a game and toy company called Sonokong. It’s a complete Korean translation of the game with English voices. The game comes in a box with a fully translated manual, which is also in color! It looks a lot better than black-and-white Japanese and US manuals I’ve seen until now. Here are some pictures of the box, the discs and the manual. Please excuse the poor picture quality, maybe someday I’ll get to scanning these properly.
Installation and initial testing
Just like the US version, the game comes on two CDs. The simple CD check is also the same, so you can install and run the game on Windows 10 and above, unlike the Japanese version which no longer works because of SafeDisc. With the Korean version, you can pass the CD check even with the US version’s CD 2.
The autorun and the installer are also fully translated.
I can’t comment on the quality of the localization since I don’t read Korean, but I can say for sure a lot of work was put into it. Unlike French, Spanish and German translations that come with the US and Japanese versions, this one translates not only all textual data, but also all menu textures, the title screen and the main menu background. But the most impressive part is the Korean font used for subtitles and Chao Garden messages. To my knowledge, this is the only official release of SADX that introduces both asset and programming changes to accommodate a language that wasn’t in the original game. It’s also executed in an interesting way, which we will get to a bit later.
As amazing as this version is, it’s still the 2004 PC port of SADX at its core, so the game has terrible graphics with no mipmaps, broken windowed mode at non-4:3 resolutions, poor music quality etc. It’s pretty hard on the eyes, especially in motion.
(From now on, I’ll be using screenshots from the US version with the Mod Loader and the Korean mod)
Korean localization and option menu differences
Korean text replaces Japanese in this version. English text is also available, although the title screen and the main menu background will always use the Korean versions of the textures. The game doesn’t let you pick Japanese voice language, so only English voices can be used. Interestingly enough, Japanese voices are still included with the game: it just doesn’t show the voice language setting in the options menu and overrides the voice language variable with English as soon as a save file is loaded. However, before a save file is loaded the game still plays the “Select a file” voice from the Japanese voices folder. To work around it, the developers replaced that voice with its English counterpart. The rest of the Japanese voices are untouched.
I’m not entirely sure, but I think such elaborate workarounds to make sure no Japanese is heard in the game may have had something to do with Korean laws regarding Japanese media, which were stricter when the game came out.
Funnily enough the texture for the option was still translated, so when you use the mod on the US version with the Mod Loader, you can select Japanese voices as if the option was intended to be there originally (although it says “Korean” instead of “Japanese”). You can also select other languages for text (not sure if you’d really need to if you’re playing this!), which the Korean version doesn’t let you do.
The localization has everything you’d expect, including character tutorials, Mini Game Collection etc.:
Reverse engineering the Korean font
The implementation of the font used in the last two screenshots is intriguing. What caught my attention initially was the large size of the FONTDATA0.BIN file, which is used for Japanese characters in the original version (1.72MB vs 759KB in the original). My first guess was that they either replaced Japanese characters with Korean ones, or added them to the end of the file with the IDs matching the Korean codepage 949.
I started inspecting the file in SADXFontEdit in SA Tools. As expected, this file contains Hangul characters used to display Korean subtitles and Chao texts. However, apart from those, the file also has some characters from other languages, including Greek, Russian, Chinese and Japanese, which amounts to a total of 20609 characters (for comparison, the original file used for Japanese has only 8836, and almost all of those are Kanji, although Cyrillic and some other characters are present as part of the Japanese codepage). On top of that, soon I realized that character IDs stored in the file matched those of the codepage 1200 (UTF-16).
Yes, you read that right. In the Korean version of the game the subtitle font is an actual Unicode font. Although it only supplies specific blocks of the Basic Multilingual Plane, in theory you could add back the other blocks to implement any language included in the plane. Back in 2020 when I first discovered this, I updated SADXFontEdit with some new functionality, including support for the Unicode layout, so when you select the correct codepage you can see the matching character next to its ID.
As we know, the US and JP versions don’t support Unicode, so how does this work? The game’s dialog texts, menu prompts etc. were compiled into the game as strings encoded with the Korean codepage 949, not Unicode. However, when the game processes text data to create a subtitle texture, it parses the characters and assigns them Unicode IDs to be used to pick graphics from the font data you see in the screenshots above.
This was where I ran into an issue I didn’t understand at first. When I first ran the game, it would either display garbage text in place of subtitles, or just crash. That was how I found out that the Korean version required the system locale (language for non-Unicode programs) to be set to Korean for subtitles to render properly.
If the game already knows it’s converting Korean to Unicode, why does it need that? Eventually I found the answer when I was looking at the game’s font related functions in the disassembly that I quickly cobbled together. The Korean version uses the function MultiByteToWideChar to convert “C strings” (in this case Korean strings in the local codepage) to UTF-16 “wide strings”. It’s an interesting solution considering that other versions have a custom function to parse characters and assign font IDs to them.
So why does it fail when the system locale isn’t set to Korean? The developers call the function with the default first argument (0), which takes the current system locale as the source codepage. As a result, Korean text is converted to Unicode as if its original codepage is whatever you have set as the language for non-Unicode programs. If the developers of the Korean version set the first argument of MultiByteToWideChar to 949, the Korean version would display subtitles correctly on any system regardless of the locale. I think a lot of people would’ve never caught this issue because the game was meant to be sold only in Korea. Anyway, we can fix this on the mod’s side.
Implementing code differences
With such changes to the font system, simply replacing FONTDATA0.BIN in a mod wouldn’t make the game support Korean, and a bunch of hacks had to be done first. One thing to do was to introduce calls to MultiByteToWideChar before letting the game parse text data. As I didn’t know the stuff I explained in the previous paragraphs, it took me a while to realize the importance of these calls, and my initial attempt to get the game to display Korean text correctly were unsuccessful. I gave up on the mod for a while, and it was only this month that I gave it another try and figured it out. Another necessary change was in the function that maps character IDs to bitmap data in the font file. There were overrides for … , 「 and 」, but other than that it was pretty straightforward.
I put together a DLL mod that replaces three text-related functions, then I replaced one Japanese hint message in Emerald Coast to test it. The Mod Loader used a hardcoded codepage for strings that are normally in Japanese, so I added global codepage overrides for all languages, which can now be used by other mods.
At first I wasn’t sure how far I could go with this, but when I finally saw the Mod Loader display Korean text, making a Korean mod started looking feasible. I was also driven by the technical challenge and the uniqueness of this version of the game, support for which required several changes to our tools, which I was happy to implement.
Extracting and importing Korean text
Speaking of technical challenges, obviously I had to somehow extract all Korean text from the game and put it into the mod. Finding it all manually was out of the question, which was part of the reason I gave up on the mod last time. However, after my experience with matching assets between different versions of the game, I was tempted by the idea of automatic splitting. Since the game still has English text, it’s possible to scan the EXE to find it, and then work up the structures to get to the start of each item. There were three types of data that had the largest amount of text: cutscene text, Tikal hints and multi-language strings such as character unlock messages, menu prompts etc. I put together a program to make the task easier. Basically it created an INI file for SA Tools’ split tool to extract the text from the Korean EXE. Of course some manual tweaks were required to ensure all addresses were correct, but the program did most of the tedious work. Cutscene text in particular required almost no touch-ups because I made it verify the text in other languages to ensure it’s the same entry in both games.
Building a mod from this data in SA Tools was a challenge. I had to add custom codepage support and fix a few bugs, but mod building in the tools is more stable now and the end result was worth it.
At this point I ran into an issue many retranslation mods had to deal with. There was some text in the game that wasn’t supported by the Mod Loader and/or the SA Tools at the time. It’s still possible to replace such text in a DLL mod, but I wanted all text replacement to be done via the Mod Loader’s INI-based system, so I had to do some work on the tools again. I added support for the following data types, which can now be replaced in mods without code:
Tikal hints that show up after you die on a boss. These are different from the hints you see in stages, and they’re stored in different ways depending on the boss, so I had to improvise a bit to make them rippable in the tools and replaceable in the Mod Loader.
Mission descriptions for Mission Mode.
Mission Mode tutorial text.
Single language strings, such as unlock prompts for Game Gear games or Metal Sonic. I added them to the SA Tools a couple years ago but not to the Mod Loader.
Multi-language strings, such as character unlock messages or “Welcome to Twinkle Circuit!”. Same as above.
Implementing Chao texts
With all that in place, all Korean text in the game was finally transferred. There was one big obstacle remaining: Chao texts. Although it loads the same font data, the Chao text system in SADX uses completely different code from the system used by subtitles and menu prompts, probably because it was copied from SA2B and duct taped with the SADX font system. So I had to do the same thing I did for the subtitles for the Chao text functions. This was probably the most difficult part of the whole mod. The functions involved with SA2B text were all difficult to hook, and even when I managed to replace them correctly I would only get crashes. It took me a long time to get a working decompilation of the functions that convert strings for the SA2B system. I’d like to thank Kell for a decompilation of these functions in the US version, which I used for reference.
Interestingly enough, the Chao system also uses wide strings internally, but all text is stored in the Japanese codepage, or Korean in this case. The game performs conversion from C strings to wide strings just the same, but the Korean version uses MultiByteToWideChar again instead of the simple parsing function used for Japanese in the original version.
After many hours of debugging I finally figured out the error in my decompiled code that was related to finding character data. In the pseudocode I got by decompiling the Korean version, the cell ID of the character data in the font was calculated by adding 0x776A to the character ID, which made it go out of range. The sign had to be flipped to make it subtract 0x8896 instead. After I fixed that, the text started working.
Now the mod was ready for testing. Out of curiosity I tried it with Dreamcast Conversion and realized that the Dreamcast Chao Garden hints were missing. Luckily they were translated in the Korean version, so I added them to SA Tools, made some changes to DC Conversion to make it more compatible with mods that replace them, and reimported the translated hints. I was happy that even the normally unused Dreamcast hints were translated.
The mod is compatible with Dreamcast Conversion, but with a few quirks. Since there are no Dreamcast menu textures in Korean, Dreamcast Conversion’s replacement of some GUI textures will be undone by this mod. This is a quick fix, and I might update it with proper compatibility eventually. There may be bugs in the mod, particularly crashes. Please send me crash dumps if you run into any.
For obvious reasons HD GUI is incompatible. If you’re interested in making HD Korean textures and an HD Korean subtitle font, please let me know.
This is the first translation mod that works by replacing the Japanese font, and the first to modify font-related code. I hope this release attracts more Korean players to SADX modding and helps other translation mods. I also think it’s truly amazing that after all these years of research and reverse engineering there are still new things about this game waiting to be discovered.
GitHub has locked me out due to Microsoft’s forced 2FA policy which I don’t want to comply with. As a result, I can no longer log in to the website to make pull requests, participate in discussions etc. However, at the moment I can still push to repos (wow, so secure huh?), so I can still work on SA Tools, the Mod Loader and other projects. They may pull the plug in the future but at least for now I can contribute to projects that are still on GitHub.
After being locked out I was able to make the following updates:
SA Tools
I added more complete support for reading and writing NJ and GJ files from various games. The update needs some testing so it will be in a separate branch for now.
SA Tools / Texture Editor
1) Unmodified texture data is no longer re-encoded unless the archive format is changed. Adding (instead of importing) textures will now add them as-is if the texture format matches the archive format (e.g. you can add DDS to PAKs or GVRs to GVMs).
2) You can choose between DDS and PNG for textures in PAKs.
3) If you open a single PVR/GVR/XVR texture, it will auto-add all other textures located in the same folder.
4) You can open and edit non-indexed PAKs (SOC folder).
5) You can save individual textures in their original format using the “Save” button.
6) Support for YUV422, Bump, 4-bit Indexed + Mipmaps, Indexed 8-bit + Mipmaps and ARGB8888 indexed PVR pixel formats.
7) Code cleanup, various bugfixes with texture encoding and decoding.
SADX Mod Loader
The game can now load textures in Gamecube GVM files without conversion. This opens up new modding possibilities, particularly for a project I’ve been contemplating to do for a while. I’ll introduce it soon.
Huge thanks to Exant, who decompiled original GVM loading code in SA2 PC, and Kell, who fixed the issues with mipmaps and paletted textures.
Today I’m releasing a new tool based on the original SA1 DLC editor. This tool has the DLC editor integrated, but its functionality expands to cover more VMS files in SA1. Eventually I would like it to support all of the formats described here, but the highlight for today’s release is the inclusion of a new Chao editor.
Like the original DLC tool, the updated editor will be included in SA Tools, which also have a big update underway. UPDATE: It’s been released and is now included in SA Tools. The Chao editor is experimental and will be updated as more information on SA1 Chao data becomes available. There is no documentation for it at the moment, although the most important data should be self explanatory.
The Chao system used in SA1 is the least researched of all Chao games. There are several Chao editors for SADX and SA2, but the only Chao editor for the Dreamcast version of SA1 was Tyro’s VMU editor that requires running on an actual Dreamcast VMU or a VMU emulator, which I found inconvenient. Besides, there have been some updates regarding SA1 Chao data (for example we now have the full SA1 Chao data struct from SADX X360 symbols), and I wanted to make use of that new knowledge to help advance SA1 Chao research. Although several aspects of it are now known better, Tyro’s notes on SA1 Chao data were one of the most important sources of information for developing the editor. I would also like to thank the Chao Island member UltimaNumber, who kindly shared the bits of information I was missing and explained how SA1 Chao data works.
The tool can import all Chao VMS data. Data produced by the Japanese version is also supported:
SA1 Chao Garden save file
Download Data (Black Market Chao)
Chao Adventure (created when you take a Chao out for a walk)
Upload Data (created when you hold Y+A as the Chao goes in the transporter)
At the moment saving is only supported for Chao Adventure and Download Data, but you can export selected Chao from the garden save file as Download Data or Chao Adventure data. There’s also an option to export files with headers used by the Japanese version of SA1.
As Chao data is rather complex and the editor is experimental, there may be some bugs, especially with saving data. Also, expect UI changes in the future.
I’d like to thank Speeps for providing a screenshot of Buddy the Chao that I used as an icon for this tool.
Apart from the GUI making it easier to edit things, this version also fixes several issues with the original tool that made it produce incorrectly aligned VMS files. Now it should be possible to modify and rebuild any of the official DLCs, or create completely new DLCs from scratch.
The tool ended up being a bit more complex than I originally anticipated, and testing it was a challenge. I’d appreciate it if you try it out and let me know on x-hax if you run into any issues. Have fun!
Following the complete SADXPC split, here’s my attempt at a near-full level/model/animation rip of SA1 Dreamcast. It includes all levels, almost all object models, the majority of motions and some data from the older split such as level object lists for most levels. When the SA Tools update is complete, it should be possible to copy this over the SADXPC split (except the object lists) and be able to load SA1 levels and objects in SADXLVL2.
This split is less complete than the SADXPC one. Here’s a list of things missing from it: -Individual landtable items are not ripped as separate files. -Various miscellaneous data, such as texlists, dialogue text etc. are not ripped. -Some levels are missing object lists. -EV files are not ripped (maybe I’ll get to that someday). -All shape motions are missing. -Some other motions may be missing. -Some object models may be missing (shouldn’t be a lot). -We don’t have source filenames for SA1 Chao assets so I created the file structure from scratch. -Some Chao animal parts may be missing or have wrong filenames. -There is some minor inconsistency with character upgrades which have Adventure Field versions. Some of those are placed in the respective Adventure Field objects folder rather than the character’s folder. -Some models and animations don’t have proper labels because I couldn’t match them with SADX assets. -Some character animations may have wrong labels.
Other than the above, it’s a complete rip of levels, models and animations that follows the structure of the new SADXPC split. To try it, extract SA1 1.005 (US, animated title screen) ISO to SA Tools\SA1\data and run SA Tools\SA1\splitSA1.bat. The output will be in SA Tools\SA1\output.
A few days ago I also added support for SA1 Gamecube .REL files that use special compression (thanks Exant for the decompression code!). The tools should now be able to work with these files, and I will likely make a split configuration for SADX Gamecube as well. It will be less complete than SA1 or SADXPC but it will focus on the assets that are different between Gamecube and PC versions. A similar split configuration may also be made for SADX Preview eventually.
Last month I wrote about a new split configuration I made for SADXPC split that ripped models from sonic.exe with a file/folder structure resembling the original source files. Recently this new configuration was expanded to include models from all SADXPC binary files, including DLL and NB files. This means we now have a 100% complete rip of all models in the entire game. And not just models, but also motions and shape motions (most of which are properly mapped to their respective models), camera actions, fog data and texture lists.
I didn’t think it was possible but we were able to identify source filenames even for models in the DLLs, for which we don’t even have labels! It turns out that most models are arranged in an alphabetical order. There were some exceptions to that, which required manual editing with some guesswork, but overall it wasn’t as unrealistic as I originally thought. I’d like to thank ItsEasyActually for identifying character-related models and motions in CHRMODELS, as well as Past objects and animations in ADV03MODELS.DLL. Also, NB files are included in this. We don’t have source filenames for them, but based on the labels used in the X360 version I was able to a at least find or guess the names for all of the objects and animations.
ItsEasyActually has been working on migrating the currently known data from the old SADXPC split to this new system. If it works out, we will be able to rebase SA Tools and its project system around the new folder structure, and the new split configuration will become the main one. I’m still not sure if this means much for mods that don’t use code. At least I’d like to help make SADXLVL2 display more object models, and I also want to make some kind of wiki for the source filenames to help people find assets easier. A long-term goal is to make DC Conversion use this system, and create a more complete split for the Dreamcast version that uses a similar folder structure.
Some statistics: there are about 12500~13000 models in the entire game (approximate number because of duplicate models), about 3450 if you don’t count level pieces. The largest models (in terms of data size) are the Egg Carrier (Sky Chase and the transformation cutscene), Sonic and Tails on the Tornado, Perfect Chaos and Chaos 6. The high poly SA1 title screen model, which is one of the biggest models in the X360 version, is not present in the 2004 PC port. Some unreferenced models are also included in this split, such as an unused Adventure Field version of Amy’s Long Hammer, the SA1 DLC Christmas tree, the cowgirl, various unused Chao toys and the unused Chao flower I wrote about some time ago.
For the last couple of months I’ve been working on a project related to SA Tools and the split tool, which I recently brought back in its original command line form, and today it’s finally ready for the public. This is a complete rip of levels, models and animations included in SADXPC’s sonic.exe that mimics the filenames and folder structure of the original source files.
This was done in multiple steps. First, Exant got me a list of source filenames from the PDB file that came with the leaked X360 prototype. The list was filtered to only include models (.NJA, .NAD and sometimes .DUP files) and animations (.NAM, DUM), and I also added levels (.c files).
Then I exported all SonicApp.exe labels using IDA and wrote a program that made lists of labels that were similar to source filenames. With some manual tweaking I was able to match all source models and animations with their labels in the database. I made the lists resemble the Dreamcast version’s file structure. For example, all Emerald Coast stuff would go in “STG01”, Station Square would be in “ADV00” etc. Both PC 2004 and the X360 prototype have most of the stuff stored in the main executable (for which we have symbols), but there’s a lot of models and animations in separate libraries too, so lists for characters and the Adventure Fields were pretty short.
Then I made it generate .INI files based on those lists and the addresses associated with the labels. Basically a bunch of split .INI files for SonicApp.exe that extracted the models and animations from it with a structure resembling the matched source assets. Now it was the time to get something similar out of the PC version.
This is the part where it got a bit more challenging. Since we didn’t have a complete split of sonic.exe, I decided to update my object scanner and make it more precise, as well as add support for motions. Motions were particularly challenging because there is no reliable way to detect the number of model parts they’re supposed to use if the model using them is unknown, which was the case with a lot of motions. But I was happy with the results since the scanner found almost all motions that I needed. Of course the models also had to be found, and to make it easier for myself I also made the scanner detect levels.
After that I had to come up with a solution to match those with the source filenames I got from the X360 version. My idea was to rely on the fact that the data should be identical between the two versions as long as it’s ripped correctly. I made some adjustments to SA Tools to make it possible to output 100% identical files from both versions as long as the model and animation data was the same, mostly by removing labels that used addresses, which were obviously different between the games. I added a bunch of undocumented command line switches to the split tool and programmed several new modes for it, including one that compared files byte by byte and made a list of file matches between two folders, and another mode that created split .INI files using lists of addresses and filename matches. This took a while but finally I was able to produce a big .INI file that contained a rough version of the file/folder structure I was looking for.
Then I split it into separate files again, and went through each file to confirm the contents. I checked for duplicates and false matches, added model part counts to all motions and assigned motions to models where possible. There were many animations that required manual editing because the models were named differently (so the tool couldn’t detect the model part count) or were missing altogether because they were supposed to be loaded from a DLL while the motion was in the EXE. For example, a lot of cutscene animations had situations like this with characters. Fixing this all was the most tedious part of the whole process, which I only finished today.
I think it was worth it. We now have a full split of all models and animations in sonic.exe (over 13000 items), including about 2720 non-level models. And, since most animations are now assigned to their models, you can preview them straight in SAMDL. Including shape motions (implemented by MainMemory earlier in the SA2 Event Viewer), which you can now also view in SAMDL.
You can try the new split configuration by downloading the latest version of SA Tools, putting sonic.exe in SATools\SADXPC\data and running splitnew.bat in the SADXPC folder.
I’ve added my DLC tool to the SA Tools suite, you can find it in the SA1 folder. It’s a simple command line tool that unpacks VMS files and rebuilds them from source assets and metadata. I want to make a GUI version of it someday, but for now I want to work on something else. The tool and the data should be self explanatory for the most part, and there’s also plenty of documentation in the readme file.
For those who want some juicy details, here’s the layout of SA1 DLC files:
* VMU HEADER
* OFFSET SIZE TYPE DESCRIPTION
* 0 16 string DLC title
* 10 32 string DLC description
* 30 16 string Application title
* 40 2 ushort Number of icons
* 42 2 ushort Animation speed
* 44 2 ushort Eyecatch type (unused)
* 46 2 ushort CRC (unused)
* 48 4 uint32 Size without the header
* 4C 20 null Reserved
* 60 32 ushort Icon palette, 16 colors
* 80 512 byte Icon graphics
*
* SECTIONS HEADER (SIZE 64 BYTES)
* 280 4 uint32 Pointer to item layout table
* 284 4 uint32 Item count
* 288 4 uint32 Pointer to string table
* 28C 4 uint32 String item count
* 290 4 uint32 Pointer to PVM
* 294 4 uint32 Number of PVMs (always 1)
* 298 4 uint32 Number of textures in the PVM
* 29C 4 uint32 Pointer to MLT
* 2A0 4 uint32 Number of MLTs (either 0 or 1)
* 2A4 4 uint32 Pointer to PRS
* 2A8 4 uint32 Number of PRSes (always 1)
* 2AC 4 uint32 Checksum
* 2B0 16 null Unused
*
* ITEM LAYOUT TABLE HEADER (SIZE 12 BYTES)
* 2C0 4 uint32 DLC ID (e.g. 504 in SONICADV_504)
* 2C4 1 byte Enable Sonic / Enable Tails
* 2C5 1 byte Enable Knuckles / Enable Gamma
* 2C6 1 byte Enable Amy / Enable Big
* 2C7 1 byte Unknown, probably unused
* 2C8 4 uint32 Regional lock
*
* ITEM LAYOUT TABLE (ARRAY BEGINS AT 0x2CC, ITEM SIZE 30 BYTES)
* 0 1 uint8 Level ID
* 1 1 uint8 Act ID
* 2 1 uint8 Scale X multiplied by 10
* 3 1 uint8 Scale Y multiplied by 10
* 4 1 uint8 Scale Z multiplied by 10
* 5 1 uint8 Rotation speed X
* 6 1 uint8 Rotation speed Y
* 7 1 uint8 Rotation speed Z
* 8 1 sint8 Item type (0: model, -128: sprite, -1: invisible)
* 9 1 uint8 Texture ID
* A 2 ushort Flags
* C 1 uint8 Object ID for collectibles or the number of objects to collect
* D 1 byte Unknown
* E 1 uint8 Message ID to show when touching the object
* F 1 uint8 Trigger distance
* 10 1 uint8 Level ID to warp or soundbank ID (8 for MLT, 15 for ADX music)
* 11 1 uint8 Act ID to warp to or sound/music ID to play
* 12 2 ushort Rotation X
* 14 2 ushort Rotation Y
* 16 2 ushort Rotation Z
* 18 2 short Position X
* 1A 2 short Position Y
* 1C 2 short Position Z
*
* DLC OBJECT FLAGS
* BIT_0 Unknown
* BIT_4 Unknown
* BIT_8 Solid
* BIT_9 Play sound
* BIT_10 Show message
* BIT_11 Hide object and disable everything except collision
* BIT_12 Warp
* BIT_13 Collectible item
* BIT_14 Timer item
* BIT_15 Starts the challenge
*
* REGIONAL LOCK BITS
* -1 Disable regional lock
* 1 Japan
* 3 US
* 4 Europe
* 7 All regions
