After I compiling Gens/GS r7 on Windows I started to implement vgm logging (like I did in MAME, MESS and openMSX). You can download the exe-file (for Windows) or the source updates (a diff file, now outdated). VGM logging is complete with a new menu entry and support for YM2612, PSG, PCM and PWM. (VGMs are logged to the "GYM dump" folder.) Almost all vgm players support the YM2612 and PSG chips. The PCM and PWM chips are in testing state and you need my vgm player (called VGMPlay) to play these files. You can download some example vgms here. Although vgm logging is almost complete, there's a little problem with the timing because the sound emulation of Gens/GS is not (but almost) sample accurate. So Kega Fusion is still recommend to log usual MegaDrive/Genesis VGMs. I'm sure GerbilSoft can find a solution. Also I want to ask, if some people that are very familiar with 32x hardware can help me to find the best way (I.e. small files) to log the PWM. (Currently I'm logging every write to the left/right channel, like it's done with YM2612 DAC.) One last note: I already optimized the PCM logging, but uncompressed files are still huge. (30 MB for 2 min, but my first 2 min PCM log had 120 MB!) I've wrote a small tool to optimize SCD PCM vgms. Download OptVgmRF exe or full source. (VGM Spec. 1.60 is here) Have fun! EDIT: Changed links due to some updates. Added OptVgmRF. EDIT2: Fixed a bug in OptVgmRF that prevented it from optimizing the RF5C68 (the Arcade chip) and added OptVgmRF source.
Holy shit, this is amazing, dude! I've been waiting for people to expand the format like this for years, and you did it without a hitch! Color me impressed. :D
Very interesting. We have had stuff set up in Project2612's backends for years just in case a development like this ever happened. If you can get it to Sample Accurate levels, then we'll definitely put it up on the recommended plugins and put it on the site, and obviously we'll then open up shop to 32x rips. A question, does this also account for the Sega CD's extra PCM channels? Not many games used the PCM channels added by it, but for "feature completeness", it should. And yes, PCM can be very large. That's why it's compressed in VGM Format 1.50. The specification should be mentioned, so that may help trimming them down further. Although I don't know how much of that would help with PWM. Definitely a nice development, though. Nice to see someone tackling it. Thumbs up, sir.
The PWM stream can be probably easily compressed with a tool similar to the existing optvgm since [most of the times] it works basically like the DAC channel of the 2612: a limited number of samples, possibly combined in a limited number of ways, so it should be possible to build up a dictionary and use it. The PCM stream of the Sega CD should be trickier to compress at least on a technical point of view, since it's not very different from a full blown module file (as far as I know). I'm not sure if there is an easy, feasible way to make it much smaller without revolutionizing the format.
I see. I'll admit I'm not as brushed up on the technical side of things as my partner in crime was, but I definitely knew it would be harder to add SCD support as opposed to 32X. Well, if worse comes to worse, there's always reinventing the wheel, if necessary. Admittedly, it may be a lot of work for too little payoff, since most SCD games used CD-DA streams, of course.
I can't say I didn't have any problems (logging PCM RAM writes was hell enough because of its asm part) but I think I got a lot of patience and a lucky hand when it comes to trial and error. btw: The accuracy of the logged VGMs is 2.8 samples. It's the accuracy of the actual emulation. Now I want to list some ideas regarding vgm compression: Let's start with DAC: I thought of a streaming driver to replace the write-DAC-and-wait-x-samples commands. (Gens KMod shows the current DAC frequency, I got the idea from there.) It would send DAC commands every x samples, depending on the frequency set. (It would also improve the quality of VGM playback with sample rates other than 44100.) PWM compression should be similar, but maybe with one data block for the left channel and one for the right one. PCM compression would be different: As I understood the PCM chip, there are 8 channels that play sounds from the chip's RAM. The chip can play the sounds at different frequencies and mixes them all. Since I'm logging RAM writes I have unmixed sounds that are a lot easier to compress than PWM or DAC. Also RAM writes can be sent some samples earlier or later without affecting the output. Accuracy is less important. I thought of using a data block with sample data (like DAC) and a command that copies xx bytes from the data block to address yy in PCM RAM. That should make the vgms a lot smaller. I personally think that it's even easier to add the PCM than the PWM.
Hi, I forgot to reply to your PM about the VGM dumping, sorry. Anyway, I will probably implement your VGM dumping functionality in Gens/GS II sometime later. (Built-in at first, and then separated into an MDP plugin once I add the necessary event handling functions.) Thanks. EDIT: Some minor issues: 64-bit: "unsigned long" is 32-bit on x86 platforms, but is 64-bit on x86_64 Linux and Mac OS X. I'll make sure this is fixed when VGM is implemented in Gens/GS II. wchar_t is 16-bit on Win32 (which is expected by the GD3 spec), but is 32-bit on other platforms. uint16_t should probably be used instead.
(EDIT I meant to post this late last night but I had to go before I could...) I'm not sure about all the details of how the 32X PWM works (I only know that it's a stream device similar to the YM2612 DAC but with a DMA mode, either mono or stereo, and with some form of internal timing system), so here's a description of the Sega CD PCM chip based on research from when I was disassembling the SMPS-PCM driver used by Sonic CD. This is a Ricoh-manufactured chip whose model number I don't remember off the top of my head. It's an 8-bit device available to the Sega CD's slave 68000 at odd addresses starting at $FF0001. It contains an eight-channel sample mixer with looping capabilities and fake stereo (I'll explain all that later). The chip takes its input data, which are 8-bit sign-magnitude samples, multiplies it by a master volume, duplicates the bytes on both the left and right speakers, and multiplies each speaker's byte by a special modifier (this is how panning is simulated) before sending the result to the mixer (it may not be in this order). The resampling in the chip is very primitive; it will just skip bytes of the input stream each cycle beeore reading the next byte. Another register determines a loop point that, in conjunction with the resampler, lets you play sustained waves larger than each channel's memory space (either $1000 or $2000 bytes; I forget). A stupidity: Ricoh used sign-magnitude format for wave data, which has both positive and negative zero, but instead of using negative zero for the loop/end of sample indicator, they use $FF — so instead of losing one possible wave value, you lose two. :specialed: So yeah nineko is half right with the module thing — it's more like the sampling engine of a MOD player.
I planned to fix that wchar_t thing before, but I forgot to do so. My vgm tagging tool already supports Linux so I'll port the code and will update the source. I may also the "long" issue - just to have better code. SegaCD PCM's name is RF5C164. There's also a chip named RF5C68 that is used in some Arcade machines (SegaSonic The Hedgehog used one). I noticed that their registers matched and tried to play SCD PCM data with the arcade chip and it worked. I guess that the SCD PCM is a console variant of the RF5C68. The same is true for the YM2612 and YM3438 variants. @Andlabs: good explanation
I've wrote a small tool called OptVgmRF. It can greatly reduce the size of vgms that use an RF-PCM chip (like Sega CD does). Download it from the first post. Code (Text): VGM RF-PCM Optimizer -------------------- File Name: SCD_CCZ_Past.vgm Step 1: Merge single memory write to larger blocks ... Step 2: Generate Block Data ... 238 Blocks created (0.12 MB, 60042x used, 0x single use). Step 3: Rewrite VGM with PCM database ... Data Compression: 43198015 -> 1445011 (3.3 %) File written. I guess it's simpler than blargg's optvgm (I wrote it within one day), but it's quite effective. Please note that the old Gens/GS and VGMPlay versions are not compatible with my last updates. (I changed some data block IDs and added a new command 0x68. I'll upload a VGM Spec. later.) You'll need to download Gens/GS VGM mod and VGMPlay again. The example files were also updated.
Excellent! 33MB to a little over 1 is a huge improvement. Do you think you'll be able to add the same optimizations to the PWM chip next?
Good to see that VGM dumping is finally being implemented for Gens/GS, as well as support for PWM and PCM. I used to use Gens more but switched to Kega due to the need for "perfect" VGM dumping. All we need now is AVI dumping and we'll have a great emulator in the end. Thanks for your work.
Just want to mention this: The 1 MB file has only 200 KB when it's compressed (vgz). Optimizing the PWM stream is a little more complicated. PWM and DAC are similar to optimize, the PCM of RF-chips needs other (easier) methods. I'm yet a little unsure how to start (splitting the left/right channel may be better than interleaving) but I know I need to write something like blargg's OptVGM. PWM doesn't have a very high priority for me because Gens/GS lacks still the accuracy for "good" vgm dumps with DAC or PWM. Btw: I just uploaded the first pack with SegaCD PCM: Sega MegaCD BIOS
Random: what exactly is your optimizer doing when it calculates "blocks?" I'm assuming it's possible to only store the original samples at one frequency and use the PCM chip registers and individual channels's memory banks to change playback rate and bank, like the original Sonic CD code does — for the record, the original SNCBNK file for Collision Chaos is 236KB (which includes the actual driver code, the SMPS data for both the music and some SFX, and the SFX samples themselves).
Actually I'm storing the original samples as they are stored in SNCBNK files. It works just like you described Sonic CD's driver. The "blocks" are based on the chip's banking system. When sound driver writes data to the PCM RAM, it fills a complete bank consecutively (usually from address 0x000 to 0xFFF - a bank has 4 KB). These consecutive writes are merged to one "block" (maybe I could also call it "bank"). -Step 1 Then it compares all blocks with each other and generates a "block database". There it writes down where it needs to write which block. -Step 2 During Step 3 the database is written and the actual blocks are replaced with pointers to the database. Note: a sample can take more than one block I hope that helps to understand the "blocks". Sorry, but I can't explain very well. Btw: Theoretically OptVgmRF could be improved. It's block size is 0x1000 (one bank), but the start-address-register has a step size of 0x0100. In practice no game seems to start a sample in the middle of a bank.
If I recall correctly, compressing an unoptimised DAC-VGM with 7-zip/LZMA would get me very small filesizes (50kb~ range) even without using optvgm. That might work for PWM VGMs too.
Okay, cool. I wonder exactly what will happen if we try to use optimization techniques on 32X VGMs considering how different games use the PWM to different extents... I don't have high hopes for any compression on After Burner Complete.
Yes, I was actually thinking about that game when, in one of my earlier posts, I said that the PWM is used like multiple DAC streams most of the times, implying that there are some games that do other things with it :P Not sure how it should be compressed, but on the other hand I am very happy to have been proven wrong about the PCM streams, so I'll just wait for ValleyBell's next move edit: here's a video in case someone isn't familiar with that game: Jump to 0:44 and listen to those guitars. Yes. Yes they are.
