You can now make your own SA1 Dreamcast DLC

I’ve added my DLC tool to the SA Tools suite, you can find it in the SA1 folder. It’s a simple command line tool that unpacks VMS files and rebuilds them from source assets and metadata. I want to make a GUI version of it someday, but for now I want to work on something else. The tool and the data should be self explanatory for the most part, and there’s also plenty of documentation in the readme file.

For those who want some juicy details, here’s the layout of SA1 DLC files:

 * 0		16		string		DLC title
 * 10		32		string		DLC description
 * 30		16		string		Application title
 * 40		2		ushort		Number of icons
 * 42		2		ushort		Animation speed
 * 44		2		ushort		Eyecatch type (unused)
 * 46		2		ushort		CRC (unused)
 * 48		4		uint32		Size without the header
 * 4C		20		null		Reserved
 * 60		32		ushort		Icon palette, 16 colors
 * 80		512		byte		Icon graphics
 * 280		4		uint32		Pointer to item layout table
 * 284		4		uint32		Item count
 * 288		4		uint32		Pointer to string table
 * 28C		4		uint32		String item count
 * 290		4		uint32		Pointer to PVM
 * 294		4		uint32		Number of PVMs (always 1)
 * 298		4		uint32		Number of textures in the PVM
 * 29C		4		uint32		Pointer to MLT
 * 2A0		4		uint32		Number of MLTs (either 0 or 1)
 * 2A4		4		uint32		Pointer to PRS
 * 2A8		4		uint32		Number of PRSes (always 1)
 * 2AC		4		uint32		Checksum
 * 2B0		16		null		Unused
 * 2C0		4		uint32		DLC ID (e.g. 504 in SONICADV_504)
 * 2C4		1		byte		Enable Sonic / Enable Tails
 * 2C5		1		byte		Enable Knuckles / Enable Gamma
 * 2C6		1		byte		Enable Amy / Enable Big
 * 2C7		1		byte		Unknown, probably unused
 * 2C8		4		uint32		Regional lock
 *  0		1		uint8		Level ID
 *  1		1		uint8		Act ID
 *  2		1		uint8		Scale X multiplied by 10
 *  3		1		uint8		Scale Y multiplied by 10
 *  4		1		uint8		Scale Z multiplied by 10
 *  5		1		uint8		Rotation speed X
 *  6		1		uint8		Rotation speed Y
 *  7		1		uint8		Rotation speed Z
 *  8		1		sint8		Item type (0: model, -128: sprite, -1: invisible)
 *  9		1		uint8		Texture ID
 *  A		2		ushort		Flags
 *  C		1		uint8		Object ID for collectibles or the number of objects to collect
 *  D		1		byte		Unknown
 *  E		1		uint8		Message ID to show when touching the object
 *  F		1		uint8		Trigger distance
 *  10		1		uint8		Level ID to warp or soundbank ID (8 for MLT, 15 for ADX music)
 *  11		1		uint8		Act ID to warp to or sound/music ID to play
 *  12		2		ushort		Rotation X
 *  14		2		ushort		Rotation Y
 *  16		2		ushort		Rotation Z
 *  18		2		short		Position X
 *  1A		2		short		Position Y
 *  1C		2		short		Position Z
 *  BIT_0	Unknown
 *  BIT_4	Unknown
 *  BIT_8	Solid
 *  BIT_9	Play sound
 *  BIT_10	Show message
 *  BIT_11	Hide object and disable everything except collision
 *  BIT_12	Warp
 *  BIT_13	Collectible item
 *  BIT_14	Timer item
 *  BIT_15	Starts the challenge
 *  -1	Disable regional lock
 *   1	Japan
 *   3	US
 *   4	Europe
 *   7	All regions

My comments on Cybershell’s video on SADX downgrades

Youtuber and streamer Cybershell, who is known for his Sonic videos from a few years back, has made a comeback with a video on SADX downgrades, which you can watch here:

It’s a great video and I recommend it to anyone interested in the subject. At the same time I feel the need to highlight several very important points that weren’t mentioned in the video:

“Dreamcast Conversion” is the name I chose for my mod. It’s a total conversion (hence the name) for SADX PC that restores Dreamcast levels, special effects, object models, textures and a lot of other things, fixes several bugs in the PC version and so on, but it doesn’t include two important aspects of the original experience: Dreamcast lighting and Dreamcast characters.

I understand that “Dreamcast Conversion” sounds like an umbrella term for all projects restoring Dreamcast stuff in SADX, which can be confusing because there are Dreamcast projects for SADX made by other people. With this in mind, I feel it is incredibly unfair to the developers of those other projects, whose authorship doesn’t belong to me. So let me clear up the situation and give shout-outs to the people whose efforts should be acknowledged just as much as (if not more than!) mine:

SonicFreak94, the developer of the Lantern Engine (palette lighting) mod. I want to make it clear that, while I helped around with research and testing SA1 lighting, the actual implementation of authentic Dreamcast lighting in SADX PC is his work. It was his incredible insight and dedication that inspired me to work on Dreamcast Conversion in the first place. I’ve contributed to several of SonicFreak94’s projects, such as SADX: Fixed Edition and the Input Mod (and he did some work on Dreamcast Conversion as well), and I am very proud to have been involved in the development of his Lantern Engine to some degree, but overall my contribution to the actual implementation of it in Direct3D was minimal, and I wouldn’t have been able to restore Dreamcast lighting on my own. So the full credit for Dreamcast lighting on PC goes to SonicFreak94, not me.

ItsEasyActually, the developer of the Dreamcast Characters mod. It’s his mod that restores original character models, and it is not part of my Dreamcast Conversion mod. While I did help with the development of this mod here and there, as well as made some minor contributions to it, it is overall a completely independent work with a ton of research and effort put into it. So we need to thank ItsEasyActually for putting all that effort in restoring Dreamcast character models in SADX PC.

MainMemory and SonicFreak94, the main developers of SADX Mod Loader and SA Tools. The Mod Loader and its mod framework make it possible to develop mods for SADX PC with different levels of complexity, from simple file replacement to custom code that rewrites parts of the game’s programming. SA Tools let us work with data from both games and are extremely useful for making comparisons or converting assets between different versions of the game. Without the Mod Loader and SA Tools it would’ve been much more difficult to develop all these mods, to the point that I probably wouldn’t have started work on Dreamcast Conversion if those tools weren’t available.

So yes, while I did have some involvement in all of these projects to some extent, it is important to understand that the restoration of Dreamcast assets in SADX PC is a combined effort of many people, and much of that effort went towards reverse engineering the game’s internals and making tools long before I came to the scene to make Dreamcast Conversion.

To sum up:
1) My Dreamcast Conversion mod is one part of the Dreamcast restoration, which includes multiple mods and is a combined effort of many people, not just me;
2) SonicFreak94 and ItsEasyActually should be credited for Lantern Engine and Dreamcast Characters respectively;
3) The Dreamcast Conversion mod is built using MainMemory and SF94’s Mod Loader framework. “PkR’s Mod Loader” is actually my Mod Installer, which installs MainMemory and SF94’s Mod Loader and a few mods.

If you want to learn more about the mods and the people behind them, I recommend TheArcadeStriker’s interviews with me, SonicFreak94 and ItsEasyActually for further reading.

Sonic Adventure: “Tikal’s challenge” DLC contest!

This release is for the original Dreamcast game, not SADX PC. A PC version of the challenge will be available at a later date.

Today I’m releasing something a bit special. This is the first fully custom* Dreamcast Sonic Adventure DLC that adds a new challenge without hacking the base game. It’s built like the official SA1 DLCs, and it makes use of SA1’s internal system to add objects and challenges to the game.

Download for emulators or transfer from PC
VMI file download for Dreamcast browser

To stay true to the spirit of official SA1 DLCs I’d like to make this a contest with a “real” (though merely symbolic) prize. To participate, simply record your playthrough of this challenge and share the video. You can play it on an emulator or on a real Dreamcast. The fastest player who beats the challenge in the shortest amount of time will get a free Sonic Adventure DX Steam key from me.

The contest will run until September 23rd, 0:00 (UTC). The deadline may be extended depending on the number of entries.
Submit your results here

To get started, you need to import the file SONICADV_515.VMS or SONICADV_515.DCI to your memory card. On PC you can use a memory card editor (such as redream’s save manager). To play it on hardware you could burn the file to a CD together with Dream Explorer, or use an SD card adapter with DreamShell. If your Dreamcast is connected to the Internet, you can download it from the revived official webpage directly from Sonic Adventure!

This DLC was made using a tool that I’ve been developing during the last several days. Once the tool is released you will be able to make your own DLCs for Sonic Adventure from scratch (within the limits of the game’s DLC system of course)!

*Technically it’s not the first edit of a DLC, but it is the first DLC that adds custom objectives and makes extensive use of the DLC system rather than edits object properties. Huge thanks to Sappharad for figuring out the integrity check for the DLCs, which made this whole thing possible.

-Darksecond for figuring out SA1 DLC and rank data encryption and posting it on ASSEMbler
-Sappharad for providing C# code to decrypt SA1 DLCs and for cracking the integrity check
-Daguar for giving me a hint on where to look for DLC item table
-Exant for making a lightweight “sitting Chao” model
-The graphics are based on Sonic Adventure (+DX) assets


Weird text in Sonic Adventure DLCs

I’m working on a tool to extract SA1 Dreamcast DLC files. While we’ve already gotten most of their data extracted and the content is recreated in the Dreamcast DLCs mod for SADX, I want to make a tool to work with DLC files directly and get everything out – item layout, text data, textures etc. I also want to make it possible to modify and rebuild DLC files.

Now why this is included after some text strings, I’m not sure. Some DLCs just have stuff inserted that makes little sense. Random garbage from memory? Some kind of weird anti-tamper measure? Who knows. Maybe I’ll figure it out eventually.

Unreferenced Chao flowers

I found this unreferenced flowers model in leaked SADX X360 symbols. It uses the Dreamcast version’s CHAO.PVM and is located in the same region as objects used in the Chao Race on DC, such as the watering can or the trumpet. What is it? Some kind of unused Chao Race object or toy? No idea.

If you want to have a look at this model, you can find it in AL_MAIN.BIN (SA1 v1.005) at 14E10C.

NB file woes

This post documents my experience with Sonic Adventure NB files and my attempts to reverse engineer the format.

NB is some kind of container format for models and animations. It’s different from container formats used in the Katana SDK, and I haven’t seen any other games use it. There are only two NB files in Sonic Adventure/DX that are used in all versions of the game: E101R.NB and EROBO.NB. As you may guess from their names, they contain models and animations for ZERO and E-101R MK II boss fights. A lot of stuff in these NB files can be found elsewhere in the game – for example, Zero’s model and animations also exist in the main binary in the 2004 version of SADX, and E101R has a cutscene model that can be ripped easily. But NB files also contain miscellaneous models for things used in these fights, such as explosions, projectiles and electric barriers, as well as some unique boss animations.

On the Gamecube the NB files are byteswapped and called E101R_GC.NB and EROBO_GC.NB, but in the PC version they are Little Endian despite the “_GC” in the filenames. Apart from Endianness the format itself is the same between all versions, and files from the Dreamcast version will work with the PC version if you simply rename them.

The NB file is composed of binary sections preceded by an 8-byte header. A section can contain models, motions or shape motions (the NB files in SA1/SADX do not contain shape motions, but I found some shape motion loading code associated with one of the section types). Each section is divided into subsections, which have their own headers and contain various types of data. For models it’s vertices, normals, materials, meshsets, UVs, model and object structures. For animations it’s rotations, positions, motion data structures and finally the motion structure itself.

So first I wanted to extract models and animations from NB files. I started with the program called SplitNB in SA Tools, which was already capable of splitting out individual binary sections from the NB file. Getting the models out of those sections was easy because the root model is always at the end of the file, so just loading a model at its address got the whole model hierarchy covered. Animations were a lot more involved, since the motion at the end of the file wouldn’t load without modifications. After a few attempts I managed to reconstruct the full animation from those individual subsections, and you can now use SplitNB in SA Tools to extract both models and animations from NB files.

The next logical step was to try and recreate those NB files from actual models and animations. That was pretty complicated since I couldn’t just write out the whole model – I needed to break all its data into sections. Same with animations, which had to have rotations and positions stored in separate sections with correct pointers to them in the motion data section.

So I managed to rebuild NB files from models and animations extracted from them previously. ZERO worked right away, but with E101R the game froze while still playing the music. After comparing original and recreated sections I found the culprit – one of the motion sections didn’t match. The original motion’s rotations section contained two rotation keyframes that my code wasn’t picking up, and the resulting section was 32 bytes smaller. But when I looked at those rotations I found out why they weren’t being picked up – they weren’t referenced anywhere in the file. Unused/leftover animation data? I don’t know, but because it’s unused, the program doesn’t include it in the extracted animation, so the rebuilt section doesn’t have it either.

But even then, why would the game freeze if the data included in the file is correct and all pointers are valid? I open the disassembly and I see this shit:

So yes, despite being able to load a pretty elaborate container format, the game uses hardcoded offsets for models in NB files, which makes any NB file rebuilding useless without additional modification of the game’s code.

Can this be worked around? Sure, if you change the offsets in code to match those in your NB file it might work. Also it will certainly work if you rebuild the file in a way that keeps original offsets for models, though it limits the possible scope of modification quite a lot. It’s disappointing because I wanted to make SA Tools support splitting and building these files for easy modification, but this hardcoded stuff is too much effort to change such specific parts of the game. I’m still going to update SA Tools with a tool to rebuild these files for lighter edits, so at least we can have that. Oh, and if you just wanted the models or animations for use in other projects, you can already have them.

This concludes my investigation into SA1/DX NB files. If you’re interested in the specifics of the format, SplitNB source code has some documentation.

First post

This is the first post on this blog, which I set up after the decision to quit Twitter.

I’m not sure whether this blog will be just like Twitter with frequent updates, or I will just dump some info here occasionally, or this will be the only post on it. We’ll see.

In the meantime you can use the top menu to have a look around.