SA1 research

Sonic Adventure patches (Better 60 FPS + drift fix)

PkR

If you’re interested in hacking Sonic Adventure, you’ve probably seen 60 FPS and widescreen codes for the Dreamcast version floating around. The problem with 60 FPS codes for SA1 is that they make cutscenes run at double speed. Here’s my understanding of why it happens and a solution for version 1.005 (US, animated title screen).

The game uses two values to control framerate and speed: the frame increment (8C754EBC) and the multiplier (8C754EC4) used in physics and other things. The frame increment controls the actual framerate (1 for 60 FPS, 2 for 30 FPS), and the multiplier speeds up or slows down the physics to match it (1 is original speed). The Dreamcast version runs at 30 FPS everywhere except menus and Twinkle Circuit. In 30 FPS mode the frame increment is set to 2, and the multiplier is also set to 2 to speed up the physics to match the reduced framerate. In menus and Twinkle Circuit both values are set to 1, so the game runs at its original speed at full framerate.

This would be all fine, but the problem is the cutscenes. They are programmed in double speed, so the game sets the frame increment to 2 and leaves the multiplier at 1 to get them to work at half speed. This is also the real reason SADX cutscenes are still capped at 30 FPS – updating them would require fixing the timings in all functions used by cutscenes to work properly at 60 FPS, which is a lot more work than simply unlocking the 60 FPS mode in levels like the developers did.

The problem with existing codes is that they don’t account for the above and simply force both values to 1, which causes cutscenes to run at double speed. To fix this, instead of changing the values directly we can do the same thing SADX developers did, and change them depending on what happens in the game. Like in SADX, there is a function that is called to set the required frame increment and multiplier. All we have to do is patch the arguments that are passed for this function when the game loads levels, and leave cutscenes alone. Here are the addresses for version 1.005 (US, animated title screen). Just change the bytes at these addresses from 2 to 1, and the game will set framerate modes like SADX. If you use Demul with Cheat Engine, replace the starting 8C with 2C.

8C04956E
8C04E662
8C054736
8C082AAE
8C09A1D2

Here’s also a code that you can add as a custom cheat for redream. It’s a regular CodeBreaker code so you could use it on other emulators or on real hardware.

cheat=0204956E 420BE501 0204E662 4F22E501 02054736 430BE501 02082AAE 420BE501 0209A1D2 D252E501

There are still two problems remaining with the 60 FPS code:

  • The hardware cannot handle the game at 60 FPS most of the time (sometimes it struggles even at 30 FPS), so you will get a slow motion effect like in older SA1 preview videos. Unfortunately this also affects emulators – for example, at the moment the game does not reach full framerate in redream. There was a similar problem with Sonic Colors, which the Wii can’t handle at 60 FPS, but that was resolved with Dolphin’s overclock feature. Maybe we’ll get that in Dreamcast emulators someday?
    UPDATE: It seems to be perfect on flycast, no slowdown at all.
  • There are glitches that happen when the game runs at 60 FPS. It’s possible that some things in the game were added after the decision to switch to 30 FPS was made, and perhaps they were only tested at 30 FPS. A lot of such glitches appear in SADX. For example, the Leon badnik almost never attacks the player and disappears too fast. There are visual glitches too, for example Tails’ tails are too fast, and the background in the tornado section in Windy Valley loses the multi-layered effect for one layer. A lot of these glitches are fixed in the Dreamcast Conversion mod for the PC version, and it’s also possible to fix them in the Dreamcast version if we really want it. Interestingly some issues that affected DX at 60 FPS don’t affect SA1 at 60 FPS – for example, the Egg Hornet engine sound glitch.

There’s another issue with SA1 on emulators related to not being able to run in a straight line with the keyboard and some controllers. You can fix this by disabling analog interpolation by writing 00 00 00 00 to 8C051278 and 8C051430. CodeBreaker code:

02051278 00000000 02051430 00000000

It’s not an ideal solution but that will do for now. A better fix may show up very soon.

Sonic Adventure DLC tool: GUI version released

PkR

I’ve updated the SA Tools with a new version of the SA1 DLC tool that lets you create and edit VMS files for SA1 on the Dreamcast.

Some documentation is available here.

Apart from the GUI making it easier to edit things, this version also fixes several issues with the original tool that made it produce incorrectly aligned VMS files. Now it should be possible to modify and rebuild any of the official DLCs, or create completely new DLCs from scratch.

The tool ended up being a bit more complex than I originally anticipated, and testing it was a challenge. I’d appreciate it if you try it out and let me know on x-hax if you run into any issues. Have fun!

New SA1 Dreamcast split

PkR

Following the complete SADXPC split, here’s my attempt at a near-full level/model/animation rip of SA1 Dreamcast. It includes all levels, almost all object models, the majority of motions and some data from the older split such as level object lists for most levels. When the SA Tools update is complete, it should be possible to copy this over the SADXPC split (except the object lists) and be able to load SA1 levels and objects in SADXLVL2.

This split is less complete than the SADXPC one. Here’s a list of things missing from it:
-Individual landtable items are not ripped as separate files.
-Various miscellaneous data, such as texlists, dialogue text etc. are not ripped.
-Some levels are missing object lists.
-EV files are not ripped (maybe I’ll get to that someday).
-All shape motions are missing.
-Some other motions may be missing.
-Some object models may be missing (shouldn’t be a lot).
-We don’t have source filenames for SA1 Chao assets so I created the file structure from scratch.
-Some Chao animal parts may be missing or have wrong filenames.
-There is some minor inconsistency with character upgrades which have Adventure Field versions. Some of those are placed in the respective Adventure Field objects folder rather than the character’s folder.
-Some models and animations don’t have proper labels because I couldn’t match them with SADX assets.
-Some character animations may have wrong labels.

Other than the above, it’s a complete rip of levels, models and animations that follows the structure of the new SADXPC split. To try it, extract SA1 1.005 (US, animated title screen) ISO to SA Tools\SA1\data and run SA Tools\SA1\splitSA1.bat. The output will be in SA Tools\SA1\output.

A few days ago I also added support for SA1 Gamecube .REL files that use special compression (thanks Exant for the decompression code!). The tools should now be able to work with these files, and I will likely make a split configuration for SADX Gamecube as well. It will be less complete than SA1 or SADXPC but it will focus on the assets that are different between Gamecube and PC versions. A similar split configuration may also be made for SADX Preview eventually.

New SADXPC split – all models ripped

PkR

Last month I wrote about a new split configuration I made for SADXPC split that ripped models from sonic.exe with a file/folder structure resembling the original source files. Recently this new configuration was expanded to include models from all SADXPC binary files, including DLL and NB files. This means we now have a 100% complete rip of all models in the entire game. And not just models, but also motions and shape motions (most of which are properly mapped to their respective models), camera actions, fog data and texture lists.

I didn’t think it was possible but we were able to identify source filenames even for models in the DLLs, for which we don’t even have labels! It turns out that most models are arranged in an alphabetical order. There were some exceptions to that, which required manual editing with some guesswork, but overall it wasn’t as unrealistic as I originally thought. I’d like to thank ItsEasyActually for identifying character-related models and motions in CHRMODELS, as well as Past objects and animations in ADV03MODELS.DLL. Also, NB files are included in this. We don’t have source filenames for them, but based on the labels used in the X360 version I was able to a at least find or guess the names for all of the objects and animations.

ItsEasyActually has been working on migrating the currently known data from the old SADXPC split to this new system. If it works out, we will be able to rebase SA Tools and its project system around the new folder structure, and the new split configuration will become the main one. I’m still not sure if this means much for mods that don’t use code. At least I’d like to help make SADXLVL2 display more object models, and I also want to make some kind of wiki for the source filenames to help people find assets easier. A long-term goal is to make DC Conversion use this system, and create a more complete split for the Dreamcast version that uses a similar folder structure.

Some statistics: there are about 12500~13000 models in the entire game (approximate number because of duplicate models), about 3450 if you don’t count level pieces. The largest models (in terms of data size) are the Egg Carrier (Sky Chase and the transformation cutscene), Sonic and Tails on the Tornado, Perfect Chaos and Chaos 6. The high poly SA1 title screen model, which is one of the biggest models in the X360 version, is not present in the 2004 PC port. Some unreferenced models are also included in this split, such as an unused Adventure Field version of Amy’s Long Hammer, the SA1 DLC Christmas tree, the cowgirl, various unused Chao toys and the unused Chao flower I wrote about some time ago.

New sonic.exe split

PkR

For the last couple of months I’ve been working on a project related to SA Tools and the split tool, which I recently brought back in its original command line form, and today it’s finally ready for the public. This is a complete rip of levels, models and animations included in SADXPC’s sonic.exe that mimics the filenames and folder structure of the original source files.

This was done in multiple steps. First, Exant got me a list of source filenames from the PDB file that came with the leaked X360 prototype. The list was filtered to only include models (.NJA, .NAD and sometimes .DUP files) and animations (.NAM, DUM), and I also added levels (.c files).

Then I exported all SonicApp.exe labels using IDA and wrote a program that made lists of labels that were similar to source filenames. With some manual tweaking I was able to match all source models and animations with their labels in the database. I made the lists resemble the Dreamcast version’s file structure. For example, all Emerald Coast stuff would go in “STG01”, Station Square would be in “ADV00” etc. Both PC 2004 and the X360 prototype have most of the stuff stored in the main executable (for which we have symbols), but there’s a lot of models and animations in separate libraries too, so lists for characters and the Adventure Fields were pretty short.

Then I made it generate .INI files based on those lists and the addresses associated with the labels. Basically a bunch of split .INI files for SonicApp.exe that extracted the models and animations from it with a structure resembling the matched source assets. Now it was the time to get something similar out of the PC version.

This is the part where it got a bit more challenging. Since we didn’t have a complete split of sonic.exe, I decided to update my object scanner and make it more precise, as well as add support for motions. Motions were particularly challenging because there is no reliable way to detect the number of model parts they’re supposed to use if the model using them is unknown, which was the case with a lot of motions. But I was happy with the results since the scanner found almost all motions that I needed. Of course the models also had to be found, and to make it easier for myself I also made the scanner detect levels.

After that I had to come up with a solution to match those with the source filenames I got from the X360 version. My idea was to rely on the fact that the data should be identical between the two versions as long as it’s ripped correctly. I made some adjustments to SA Tools to make it possible to output 100% identical files from both versions as long as the model and animation data was the same, mostly by removing labels that used addresses, which were obviously different between the games. I added a bunch of undocumented command line switches to the split tool and programmed several new modes for it, including one that compared files byte by byte and made a list of file matches between two folders, and another mode that created split .INI files using lists of addresses and filename matches. This took a while but finally I was able to produce a big .INI file that contained a rough version of the file/folder structure I was looking for.

Then I split it into separate files again, and went through each file to confirm the contents. I checked for duplicates and false matches, added model part counts to all motions and assigned motions to models where possible. There were many animations that required manual editing because the models were named differently (so the tool couldn’t detect the model part count) or were missing altogether because they were supposed to be loaded from a DLL while the motion was in the EXE. For example, a lot of cutscene animations had situations like this with characters. Fixing this all was the most tedious part of the whole process, which I only finished today.

I think it was worth it. We now have a full split of all models and animations in sonic.exe (over 13000 items), including about 2720 non-level models. And, since most animations are now assigned to their models, you can preview them straight in SAMDL. Including shape motions (implemented by MainMemory earlier in the SA2 Event Viewer), which you can now also view in SAMDL.

You can try the new split configuration by downloading the latest version of SA Tools, putting sonic.exe in SATools\SADXPC\data and running splitnew.bat in the SADXPC folder.

image

Alternative music in XMAS99 DLC

PkR

There are two NiGHTS-related music tracks on Sonic Adventure GD-ROM. One is called NIGHTS_A, which is the a capella cover of “Dreams Dreams”. The other is called NIGHTS_S, which is another mix of “Dreams Dreams” called “Sweet Mix in the Holy Night”. The A version is the one you hear with the Christmas 99 DLC when you touch any of the Christmas trees, the other one you don’t get to hear ingame normally but there’s a bit more to it than the song being unused.

A while back I found out that the S version is actually referenced in the Christmas 99 DLC, but the object that plays the song never loads because it has the “wrong” level ID, and the one that plays the A version gets loaded in its place because it has the “right” level ID. In the game’s internal level list Station Square has the ID of 26, while the objects playing the S version use 27 and 28. It’s pretty clear that at some point in development the game had the day, evening and night versions of Station Square use different internal IDs, even though the final game only uses 26. Even the “Easy Menu” in SADX Preview prototype refers to 27 and 28 as evening and night versions respectively.

The XMAS99 DLC places four Christmas trees around Station Square. If you look at its object placement (which you can do with the DLC Tool in SA Tools), you can see that for each of those trees there are “evening” and “night” versions that are supposed to load in levels 27 and 28. One of the parameters in those objects is the music track ID to play. The regular “day” trees use 68, which corresponds to NIGHTS_A, but some of the “evening” and “night” trees use 70, which is NIGHTS_S. For example, this one in the City Hall area:

While it is trivial to switch the IDs around and make these trees play the S version at all times, I don’t know of a way to restore the seemingly intended functionality with the trees playing different songs at different times of day. The old Dreamcast DLCs mod for SADX had this behavior hardcoded, but the new mod loads objects in a way a bit more similar to the Dreamcast version, so it doesn’t work for the same reason it doesn’t work in the original. I’ll probably just make this tree play the S version because why not?

There’s one more interesting thing about this DLC and unused NiGHTS songs in SA1. We’ve seen both music IDs 68 and 70 used, but what about 69? While it isn’t referenced in the DLC, we know from the PC version that there was supposed to be yet another NiGHTS song called NIGHTS_K, which would be using this ID. As no version of the game has this file, we can’t say for sure what was supposed to be. The Kids version of “Dreams, Dreams”? The Karaoke version? “Know Thyself!” maybe? To me, the Karaoke version makes the most sense, but we can only guess.

The updated DLCs mod for SADX PC is coming along well. I rewrote the entire mod from scratch and made it work in a way that resembles the functionality used in the original game. It’s also partially compatible with original data from SA1 DLC files provided that you convert it first using the DLC tool and replace some of the stuff that isn’t supported by the PC version, such as MLT soundbanks. Model and metadata files are fully compatible. So in the future if you decide to make a custom DLC for the Dreamcast version, you will be able to prototype it in SADX PC first using the new DLC mod, which makes object placement easier.

Just… why

PkR

I’ve been wondering for a long time how SA1 draws its “Now Saving” and DLC timer strings. Apart from being italicized (and looking glitchy with letters overlapping each other) the font looks thicker than the font used in debug menus, and the 0 character looks different, and I started thinking it was a different font altogether.

Today Exant figured it out, and the answer to all this is just… Weird. It’s the same font, but there’s code that… replaces the zeroes in text with the letter O. The rest is a lot more believable: the font looks thicker because the letters are stretched horizontally, and the italics are done by offsetting the top right X coordinate of the letter’s rectangle.

I’ll be updating DC Conversion and the DLCs mod to improve the look of the font, though I also want to rework the DLCs mod from the ground up. I want to make it possible to load the data produced by the SA1 DLC tool I made for SA Tools. That way you’ll be able to convert DLCs between SA1 and SADX. There’s some fundamental incompatibility with sounds and music, which I won’t be able to resolve in a while, so it will probably be only partially compatible, but anyway I’ll save that for another post.

You can now make your own SA1 Dreamcast DLC

PkR

I’ve added my DLC tool to the SA Tools suite, you can find it in the SA1 folder. It’s a simple command line tool that unpacks VMS files and rebuilds them from source assets and metadata. I want to make a GUI version of it someday, but for now I want to work on something else. The tool and the data should be self explanatory for the most part, and there’s also plenty of documentation in the readme file.

For those who want some juicy details, here’s the layout of SA1 DLC files:

 * VMU HEADER
 * OFFSET	SIZE		TYPE		DESCRIPTION
 * 0		16		string		DLC title
 * 10		32		string		DLC description
 * 30		16		string		Application title
 * 40		2		ushort		Number of icons
 * 42		2		ushort		Animation speed
 * 44		2		ushort		Eyecatch type (unused)
 * 46		2		ushort		CRC (unused)
 * 48		4		uint32		Size without the header
 * 4C		20		null		Reserved
 * 60		32		ushort		Icon palette, 16 colors
 * 80		512		byte		Icon graphics
 * 
 * SECTIONS HEADER (SIZE 64 BYTES) 
 * 280		4		uint32		Pointer to item layout table
 * 284		4		uint32		Item count
 * 288		4		uint32		Pointer to string table
 * 28C		4		uint32		String item count
 * 290		4		uint32		Pointer to PVM
 * 294		4		uint32		Number of PVMs (always 1)
 * 298		4		uint32		Number of textures in the PVM
 * 29C		4		uint32		Pointer to MLT
 * 2A0		4		uint32		Number of MLTs (either 0 or 1)
 * 2A4		4		uint32		Pointer to PRS
 * 2A8		4		uint32		Number of PRSes (always 1)
 * 2AC		4		uint32		Checksum
 * 2B0		16		null		Unused
 * 
 * ITEM LAYOUT TABLE HEADER (SIZE 12 BYTES)
 * 2C0		4		uint32		DLC ID (e.g. 504 in SONICADV_504)
 * 2C4		1		byte		Enable Sonic / Enable Tails
 * 2C5		1		byte		Enable Knuckles / Enable Gamma
 * 2C6		1		byte		Enable Amy / Enable Big
 * 2C7		1		byte		Unknown, probably unused
 * 2C8		4		uint32		Regional lock
 *
 * ITEM LAYOUT TABLE (ARRAY BEGINS AT 0x2CC, ITEM SIZE 30 BYTES)
 *  0		1		uint8		Level ID
 *  1		1		uint8		Act ID
 *  2		1		uint8		Scale X multiplied by 10
 *  3		1		uint8		Scale Y multiplied by 10
 *  4		1		uint8		Scale Z multiplied by 10
 *  5		1		uint8		Rotation speed X
 *  6		1		uint8		Rotation speed Y
 *  7		1		uint8		Rotation speed Z
 *  8		1		sint8		Item type (0: model, -128: sprite, -1: invisible)
 *  9		1		uint8		Texture ID
 *  A		2		ushort		Flags
 *  C		1		uint8		Object ID for collectibles or the number of objects to collect
 *  D		1		byte		Unknown
 *  E		1		uint8		Message ID to show when touching the object
 *  F		1		uint8		Trigger distance
 *  10		1		uint8		Level ID to warp or soundbank ID (8 for MLT, 15 for ADX music)
 *  11		1		uint8		Act ID to warp to or sound/music ID to play
 *  12		2		ushort		Rotation X
 *  14		2		ushort		Rotation Y
 *  16		2		ushort		Rotation Z
 *  18		2		short		Position X
 *  1A		2		short		Position Y
 *  1C		2		short		Position Z
 *
 * DLC OBJECT FLAGS
 *  BIT_0	Unknown
 *  BIT_4	Unknown
 *  BIT_8	Solid
 *  BIT_9	Play sound
 *  BIT_10	Show message
 *  BIT_11	Hide object and disable everything except collision
 *  BIT_12	Warp
 *  BIT_13	Collectible item
 *  BIT_14	Timer item
 *  BIT_15	Starts the challenge
 *
 * REGIONAL LOCK BITS
 *  -1	Disable regional lock
 *   1	Japan
 *   3	US
 *   4	Europe
 *   7	All regions

Sonic Adventure: “Tikal’s challenge” DLC contest!

PkR

This release is for the original Dreamcast game, not SADX PC. A PC version of the challenge will be available at a later date.

Today I’m releasing something a bit special. This is the first fully custom* Dreamcast Sonic Adventure DLC that adds a new challenge without hacking the base game. It’s built like the official SA1 DLCs, and it makes use of SA1’s internal system to add objects and challenges to the game.

Download for emulators or transfer from PC
VMI file download for Dreamcast browser

To stay true to the spirit of official SA1 DLCs I’d like to make this a contest with a “real” (though merely symbolic) prize. To participate, simply record your playthrough of this challenge and share the video. You can play it on an emulator or on a real Dreamcast. The fastest player who beats the challenge in the shortest amount of time will get a free Sonic Adventure DX Steam key from me.

The contest will run until September 23rd, 0:00 (UTC). The deadline may be extended depending on the number of entries.
Submit your results here

To get started, you need to import the file SONICADV_515.VMS or SONICADV_515.DCI to your memory card. On PC you can use a memory card editor (such as redream’s save manager). To play it on hardware you could burn the file to a CD together with Dream Explorer, or use an SD card adapter with DreamShell. If your Dreamcast is connected to the Internet, you can download it from the revived official webpage directly from Sonic Adventure!

This DLC was made using a tool that I’ve been developing during the last several days. Once the tool is released you will be able to make your own DLCs for Sonic Adventure from scratch (within the limits of the game’s DLC system of course)!

*Technically it’s not the first edit of a DLC, but it is the first DLC that adds custom objectives and makes extensive use of the DLC system rather than edits object properties. Huge thanks to Sappharad for figuring out the integrity check for the DLCs, which made this whole thing possible.

Credits:
-Darksecond for figuring out SA1 DLC and rank data encryption and posting it on ASSEMbler
-Sappharad for providing C# code to decrypt SA1 DLCs and for cracking the integrity check
-Daguar for giving me a hint on where to look for DLC item table
-Exant for making a lightweight “sitting Chao” model
-The graphics are based on Sonic Adventure (+DX) assets

Enjoy!