Discontinuing HD GUI

This is a quick note to let you know that I will no longer provide updates for the HD GUI mod. The repository has been archived. If you are interested in continuing the mod’s development, feel free to unarchive it and/or make a fork of it.

The fascinating Korean version of SADX PC and release of the Korean mod

TLDR version: There was a Korean version of SADX PC released back in 2004. I ported the Korean translation to the Mod Loader, so you can now play SADX in Korean if you install this mod. The mod is compatible with Dreamcast Conversion and Dreamcast DLC but incompatible with HD GUI.

Four years ago I got my hands on a somewhat rare version of SADX PC. I was completely unaware of its existence until I accidentally ran into a listing on an auction, so I got it right away. I’ve mentioned it before in a previous post, and today’s mod release brings the occasion to finally talk about it again.

Introduction and game package

This version was released in July 2004 in Korea by a game and toy company called Sonokong. It’s a complete Korean translation of the game with English voices. The game comes in a box with a fully translated manual, which is also in color! It looks a lot better than black-and-white Japanese and US manuals I’ve seen until now. Here are some pictures of the box, the discs and the manual. Please excuse the poor picture quality, maybe someday I’ll get to scanning these properly.

Installation and initial testing

Just like the US version, the game comes on two CDs. The simple CD check is also the same, so you can install and run the game on Windows 10 and above, unlike the Japanese version which no longer works because of SafeDisc. With the Korean version, you can pass the CD check even with the US version’s CD 2.

The autorun and the installer are also fully translated.

I can’t comment on the quality of the localization since I don’t read Korean, but I can say for sure a lot of work was put into it. Unlike French, Spanish and German translations that come with the US and Japanese versions, this one translates not only all textual data, but also all menu textures, the title screen and the main menu background. But the most impressive part is the Korean font used for subtitles and Chao Garden messages. To my knowledge, this is the only official release of SADX that introduces both asset and programming changes to accommodate a language that wasn’t in the original game. It’s also executed in an interesting way, which we will get to a bit later.

As amazing as this version is, it’s still the 2004 PC port of SADX at its core, so the game has terrible graphics with no mipmaps, broken windowed mode at non-4:3 resolutions, poor music quality etc. It’s pretty hard on the eyes, especially in motion.

(From now on, I’ll be using screenshots from the US version with the Mod Loader and the Korean mod)

Korean localization and option menu differences

Korean text replaces Japanese in this version. English text is also available, although the title screen and the main menu background will always use the Korean versions of the textures. The game doesn’t let you pick Japanese voice language, so only English voices can be used. Interestingly enough, Japanese voices are still included with the game: it just doesn’t show the voice language setting in the options menu and overrides the voice language variable with English as soon as a save file is loaded. However, before a save file is loaded the game still plays the “Select a file” voice from the Japanese voices folder. To work around it, the developers replaced that voice with its English counterpart. The rest of the Japanese voices are untouched.

I’m not entirely sure, but I think such elaborate workarounds to make sure no Japanese is heard in the game may have had something to do with Korean laws regarding Japanese media, which were stricter when the game came out.

Funnily enough the texture for the option was still translated, so when you use the mod on the US version with the Mod Loader, you can select Japanese voices as if the option was intended to be there originally (although it says “Korean” instead of “Japanese”). You can also select other languages for text (not sure if you’d really need to if you’re playing this!), which the Korean version doesn’t let you do.

The localization has everything you’d expect, including character tutorials, Mini Game Collection etc.:

Reverse engineering the Korean font

The implementation of the font used in the last two screenshots is intriguing. What caught my attention initially was the large size of the FONTDATA0.BIN file, which is used for Japanese characters in the original version (1.72MB vs 759KB in the original). My first guess was that they either replaced Japanese characters with Korean ones, or added them to the end of the file with the IDs matching the Korean codepage 949.

I started inspecting the file in SADXFontEdit in SA Tools. As expected, this file contains Hangul characters used to display Korean subtitles and Chao texts. However, apart from those, the file also has some characters from other languages, including Greek, Russian, Chinese and Japanese, which amounts to a total of 20609 characters (for comparison, the original file used for Japanese has only 8836, and almost all of those are Kanji, although Cyrillic and some other characters are present as part of the Japanese codepage). On top of that, soon I realized that character IDs stored in the file matched those of the codepage 1200 (UTF-16).

Yes, you read that right. In the Korean version of the game the subtitle font is an actual Unicode font. Although it only supplies specific blocks of the Basic Multilingual Plane, in theory you could add back the other blocks to implement any language included in the plane. Back in 2020 when I first discovered this, I updated SADXFontEdit with some new functionality, including support for the Unicode layout, so when you select the correct codepage you can see the matching character next to its ID.

As we know, the US and JP versions don’t support Unicode, so how does this work? The game’s dialog texts, menu prompts etc. were compiled into the game as strings encoded with the Korean codepage 949, not Unicode. However, when the game processes text data to create a subtitle texture, it parses the characters and assigns them Unicode IDs to be used to pick graphics from the font data you see in the screenshots above.

This was where I ran into an issue I didn’t understand at first. When I first ran the game, it would either display garbage text in place of subtitles, or just crash. That was how I found out that the Korean version required the system locale (language for non-Unicode programs) to be set to Korean for subtitles to render properly.

If the game already knows it’s converting Korean to Unicode, why does it need that? Eventually I found the answer when I was looking at the game’s font related functions in the disassembly that I quickly cobbled together. The Korean version uses the function MultiByteToWideChar to convert “C strings” (in this case Korean strings in the local codepage) to UTF-16 “wide strings”. It’s an interesting solution considering that other versions have a custom function to parse characters and assign font IDs to them.

So why does it fail when the system locale isn’t set to Korean? The developers call the function with the default first argument (0), which takes the current system locale as the source codepage. As a result, Korean text is converted to Unicode as if its original codepage is whatever you have set as the language for non-Unicode programs. If the developers of the Korean version set the first argument of MultiByteToWideChar to 949, the Korean version would display subtitles correctly on any system regardless of the locale. I think a lot of people would’ve never caught this issue because the game was meant to be sold only in Korea. Anyway, we can fix this on the mod’s side.

Implementing code differences

With such changes to the font system, simply replacing FONTDATA0.BIN in a mod wouldn’t make the game support Korean, and a bunch of hacks had to be done first. One thing to do was to introduce calls to MultiByteToWideChar before letting the game parse text data. As I didn’t know the stuff I explained in the previous paragraphs, it took me a while to realize the importance of these calls, and my initial attempt to get the game to display Korean text correctly were unsuccessful. I gave up on the mod for a while, and it was only this month that I gave it another try and figured it out. Another necessary change was in the function that maps character IDs to bitmap data in the font file. There were overrides for , and , but other than that it was pretty straightforward.

I put together a DLL mod that replaces three text-related functions, then I replaced one Japanese hint message in Emerald Coast to test it. The Mod Loader used a hardcoded codepage for strings that are normally in Japanese, so I added global codepage overrides for all languages, which can now be used by other mods.

At first I wasn’t sure how far I could go with this, but when I finally saw the Mod Loader display Korean text, making a Korean mod started looking feasible. I was also driven by the technical challenge and the uniqueness of this version of the game, support for which required several changes to our tools, which I was happy to implement.

Extracting and importing Korean text

Speaking of technical challenges, obviously I had to somehow extract all Korean text from the game and put it into the mod. Finding it all manually was out of the question, which was part of the reason I gave up on the mod last time. However, after my experience with matching assets between different versions of the game, I was tempted by the idea of automatic splitting. Since the game still has English text, it’s possible to scan the EXE to find it, and then work up the structures to get to the start of each item. There were three types of data that had the largest amount of text: cutscene text, Tikal hints and multi-language strings such as character unlock messages, menu prompts etc. I put together a program to make the task easier. Basically it created an INI file for SA Tools’ split tool to extract the text from the Korean EXE. Of course some manual tweaks were required to ensure all addresses were correct, but the program did most of the tedious work. Cutscene text in particular required almost no touch-ups because I made it verify the text in other languages to ensure it’s the same entry in both games.

Building a mod from this data in SA Tools was a challenge. I had to add custom codepage support and fix a few bugs, but mod building in the tools is more stable now and the end result was worth it.

At this point I ran into an issue many retranslation mods had to deal with. There was some text in the game that wasn’t supported by the Mod Loader and/or the SA Tools at the time. It’s still possible to replace such text in a DLL mod, but I wanted all text replacement to be done via the Mod Loader’s INI-based system, so I had to do some work on the tools again. I added support for the following data types, which can now be replaced in mods without code:

  • Tikal hints that show up after you die on a boss. These are different from the hints you see in stages, and they’re stored in different ways depending on the boss, so I had to improvise a bit to make them rippable in the tools and replaceable in the Mod Loader.
  • Mission descriptions for Mission Mode.
  • Mission Mode tutorial text.
  • Single language strings, such as unlock prompts for Game Gear games or Metal Sonic. I added them to the SA Tools a couple years ago but not to the Mod Loader.
  • Multi-language strings, such as character unlock messages or “Welcome to Twinkle Circuit!”. Same as above.

Implementing Chao texts

With all that in place, all Korean text in the game was finally transferred. There was one big obstacle remaining: Chao texts. Although it loads the same font data, the Chao text system in SADX uses completely different code from the system used by subtitles and menu prompts, probably because it was copied from SA2B and duct taped with the SADX font system. So I had to do the same thing I did for the subtitles for the Chao text functions. This was probably the most difficult part of the whole mod. The functions involved with SA2B text were all difficult to hook, and even when I managed to replace them correctly I would only get crashes. It took me a long time to get a working decompilation of the functions that convert strings for the SA2B system. I’d like to thank Kell for a decompilation of these functions in the US version, which I used for reference.

Interestingly enough, the Chao system also uses wide strings internally, but all text is stored in the Japanese codepage, or Korean in this case. The game performs conversion from C strings to wide strings just the same, but the Korean version uses MultiByteToWideChar again instead of the simple parsing function used for Japanese in the original version.

After many hours of debugging I finally figured out the error in my decompiled code that was related to finding character data. In the pseudocode I got by decompiling the Korean version, the cell ID of the character data in the font was calculated by adding 0x776A to the character ID, which made it go out of range. The sign had to be flipped to make it subtract 0x8896 instead. After I fixed that, the text started working.

Now the mod was ready for testing. Out of curiosity I tried it with Dreamcast Conversion and realized that the Dreamcast Chao Garden hints were missing. Luckily they were translated in the Korean version, so I added them to SA Tools, made some changes to DC Conversion to make it more compatible with mods that replace them, and reimported the translated hints. I was happy that even the normally unused Dreamcast hints were translated.

Mod download, compatibility and conclusion

SA Mod Manager 1-click install
Manual download

The mod is compatible with Dreamcast Conversion, but with a few quirks. Since there are no Dreamcast menu textures in Korean, Dreamcast Conversion’s replacement of some GUI textures will be undone by this mod. This is a quick fix, and I might update it with proper compatibility eventually. There may be bugs in the mod, particularly crashes. Please send me crash dumps if you run into any.

For obvious reasons HD GUI is incompatible. If you’re interested in making HD Korean textures and an HD Korean subtitle font, please let me know.

This is the first translation mod that works by replacing the Japanese font, and the first to modify font-related code. I hope this release attracts more Korean players to SADX modding and helps other translation mods. I also think it’s truly amazing that after all these years of research and reverse engineering there are still new things about this game waiting to be discovered.

The mod’s source code is located here.

SADX Mod Installer update

The SADX Mod Installer was discontinued back in 2022. However, with the arrival of the new SA Mod manager and retirement of the original SADX Mod Manager I felt it was necessary to update it. Many people are still using it, and Linux/Steam Deck tutorials seem to rely on it. Although I want to retire the installer as soon as possible, I made the following update to make it easier for people using it right now:

  • If you’re running Windows 7 or above, it will install the new SA Mod Manager (auto picking x86 or x64). If you’re running Windows XP, it will install the legacy SADX Mod Manager.
  • .NET 7.0 Desktop (x86 or x64) is installed automatically on Windows 7 and above.
  • The installer was made compatible with Windows XP. All tools, including Steam conversion, are completely functional on XP. Online functionality will not work on XP and 7 due to outdated certificates, but it might work if the system has correct certificates.
  • The Steam conversion process has been altered to rely on patches. Now the installer doesn’t supply the EXE or DLLs from the 2004 version, instead it patches the Steam version to turn it into the 2004 US version using a 5MB diff file. This was made possible thanks to HDiffPatch. Best Buy, EU original, Sonic PC Collection and Korean versions of SADX are also patchable.

The installer still doesn’t support the new configuration format introduced in the new Mod Manager, and I think it will probably stay that way. However, the new Mod Manager picks up the old format settings generated by the installer so this should be no problem to the players.

At some point in the future the new Mod Manager will handle the Steam conversion, and the installer will no longer be necessary. Some of its features will be implemented in the new Mod Manager or through other means: for example, Guide Mode could be done as a web page with 1-click install links.

Native Steam savegame support in SADX 2004

Previously you had to use BetterSADX or the SADX Mod Installer to transfer savegames to the 2004 version, but soon it will be possible to load the Steam version’s saved games in the Mod Loader. You will be able to load files directly from their original location, so no transfer is required whatsoever. If you play a Steam saved game in the modded 2004 version, the original Steam version will also be able to load it, so you can play the same file on both versions. The Chao Garden save is also supported: if you load a saved game located in the Steam folder, it will use the Steam version’s garden file. As you can see in the screenshot, custom filenames (ANSI/English only) are also possible, so your saved games no longer have to be named “SonicDX??” for the game to recognize them.

The format used for saved games is almost identical between the two versions, but there are a couple differences that make them incompatible. You can load 2004 saves in the Steam version if you rename them, but not the other way around. The files created by the Steam version are 16 bytes bigger (1408 bytes vs 1392 in the old port), and 12 of these extra bytes are used to store the file creation date and time. At the beginning of the file there is a checksum, and both versions use the same algorithm which calculates the checksum value based on the contents of the whole file. The reason the 2004 version won’t load Steam saved games is because the game only accounts for 1392 bytes when reading the file and calculating the checksum, which doesn’t match because it doesn’t include those extra 16 bytes in the calculation. The Steam version removes the checksum check on the file select screen so it recognizes saved games from the 2004 version, although the date and time might look wrong. When progress is saved in a file that originally came from the Steam version, the Mod Loader writes the current date and time into the file so it should display correctly in the Steam version as well.

Leaving GitHub

Microsoft, who acquired GitHub 5 years ago, are demonstrating their true intentions regarding the future of the site that used to be the largest hub of free and open source community projects.

I was a user of GitHub before Microsoft acquired it, and they have no moral right to impose their new rules and threaten to lock me out of my work if I don’t obey. You can debate the security, necessity and efficacy of 2FA in general or for GitHub in particular, but to sum up my thoughts on this measure:
– I’m not going to use any form of 2FA for my account on GitHub;
– I find forcing 2FA on long-term users, AND locking out those who don’t want it, unacceptable.

So, I have decided to leave GitHub permanently. I will not be coming back to GitHub even if they reverse the decision (though obviously they won’t).

Among other projects, the following SADX mods and tools have had their source code relocated to GitLab:
Dreamcast Conversion
Dreamcast DLC
Sound Overhaul
Debug Mode
SADX Style Water
Time of Day
SADX Mod Installer

As I am leaving GitHub, I will no longer be able to contribute to the following projects on the X-Hax organization:
SADX Mod Loader / Mod Loader Common
SA Tools and SA Tools Wiki
SA Tools (research)
SADX Modding Wiki
Decompilation projects

I will also be losing the ability to contribute (at least directly) to the following projects:
Lantern Engine
Input Mod
SADX: Fixed Edition
Onion Blur

Depending on what kind of “restrictions” Microsoft put on my account, I may also become unable to report issues or participate in discussions in the above projects after October 6th, 2023.

Leaving is not easy, and losing touch with the community and projects I’ve put a lot of personal time into building and improving sure hurts. Still, I hope you understand and accept my decision.

Thank you for following my work all this time.

SADX Debug Mode version 2.0

Today I’m releasing an update for the Debug Mode mod that adds a free camera mode.

Using the free camera mode

Press Y on the keyboard to enable free camera. When the free camera mode is enabled, you can use the mouse to move the camera around. The control scheme is similar to the one used in 3D editors in SA Tools.

  • Move the mouse to rotate the camera.
  • Hold Left Shift and move the mouse to move the camera.
  • Hold Control and move the mouse up and down to zoom in and out.
  • Press both Left Shift and Control to lock the camera in place. Press them again to unlock it.
  • Press Numpad + and Numpad – to change camera speed.
  • Press Y again to re-enable vanilla camera. It will remain locked in place until the character touches a camera trigger.

I would like to thank Speeps for providing the source code of an unreleased mod with the the original free camera code, which became the basis for the free camera mode in this mod.

Dreamcast DLC mod release

The new version of the Dreamcast DLC mod is now out! You can use this mod to play SA1 DC downloadable content in SADX PC, as well as develop your own custom downloads that will work both in the mod and even in the Dreamcast version provided that you convert it using SA Tools first. Also, this version includes Tikal’s challenge, a custom DLC created previously for the Dreamcast version.

Get the mod here or simply update the previous version through the Mod Loader. If you get a “directory not empty” error while updating the mod, just hit cancel and it’ll work fine. I don’t know why it happens.

Here’s an excerpt from readme/release notes:
-SonicFreak94 for the code to play the SEGA/Sonic Team voice
-Darksecond for figuring out SA1 DLC and rank data encryption and posting it on ASSEMbler
-Sappharad for providing C# code to decrypt SA1 DLCs and for cracking the integrity check
-Daguar for giving me a hint on where to look for DLC item table
-Exant for making a lightweight “sitting Chao” model for the Tikal DLC
-Moopthehedgehog for the rediscovered Kadomatsu DLC
-TheLegendOfXela for high quality music and sound rips
-LastBreath for beta testing

-The mod has been rewritten from scratch to work more like the original game’s DLC system
-Text strings in all languages used in original DLC content have been preserved
-DLC content can now load in Adventure, Trial or Mission Mode
-Added Tikal’s challenge
-Lighting accuracy updates
-Custom codepage support: add system\dlc\codepage.ini with “FrenchCodepage=….” etc. to make French, Spanish or German strings use a codepage other than 1252
-Famitsu and QUO challenges no longer replace Station Square camera files with their counterparts from the original 1998 release of SA1
-Twinkle Circuit options for characters have been removed – you can now use the “Mercury GP” menu to go to any course with any character
-The “Alternative challenge timer GUI” option has been removed in favor of a more accurate SA1-like timer
-Seasonal DLC options have been rearranged
-Workarounds from the old mod to prevent DLC content from interfering with story progression have not been reimplemented

New Twinkle Circuit menu preview

I’m working on a complete rewrite of the Dreamcast DLC mod to practice making something that uses C++ classes. With the new version you will be able to modify original SA1 DLCs and create completely new DLC content that will work in both SADX PC and SA1 Dreamcast with a simple conversion (except custom sounds and music which aren’t compatible).

Some features of the old mod won’t make it to this version. For example, all Twinkle Circuit settings in the old mod will have to go. To compensate for it I made this little menu that will show up when you enter the special gate that appears when you enable the Samba GP DLC. The look may or may not be final, I’m still messing around with its design. I’m a bit limited in how much I can do in it.