don't click here

ZOMG Savestate Format

Discussion in 'Engineering & Reverse Engineering' started by GerbilSoft, Aug 25, 2010.

  1. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    I've been working on a new savestate format for Gens/GS II called "ZOMG", or "Zipped Original Memory from Genesis". Unlike previous savestate formats, which were undocumented binary blobs, ZOMG is a fully-documented Zip archive.

    The current specifications are available at
    <a href="http://gs_server.gerbilsoft.ddns.info/cgi-bin/gitweb.cgi?p=gens-gs-ii.git;a=blob;f=doc/ZOMG.txt;hb=HEAD" target="_blank">http://gs_server.gerbilsoft.ddns.info/cgi-...OMG.txt;hb=HEAD</a>

    Components implemented in the specification thus far:
    • VDP registers (24 user-visible registers; does NOT include address counter and other internal registers.)
    • VRam
    • CRam
    • PSG
    • Z80 memory
    • Z80 registers

    MD-specific components:
    • VSRam
    • YM2612 (registers only; timers need to be added)
    • M68K memory
    • M68K registers
    • I/O ports ($A10000-$A1001F range)
    • Z80 control logic (BUSREQ, RESET, banking)

    I'd like some comments on the current draft. In particular, MD/Z80_ctrl.bin currently uses "1" to indicate the Z80 is in RESET, whereas the actual value used for resetting the Z80 is "0". (Gens itself, and the Genecyst savestate format, use "1" to indicate RESET.) I'm not sure if I should change it to match the actual system or keep it so it matches Gens internally and also the old savestate format.

    EDIT: I switched BUSREQ and RESET to use the values used on the actual MD instead of the Genecyst values. (BUSREQ: 0 == Z80 has bus, 1 == M68K has bus; RESET: 0 == Z80 is RESET, 1 == Z80 is running.) This is opposite of what Gens/GS II uses internally, so I'll probably change Gens/GS II's internal representation later.
     
  2. Will this format support 32X and Mega CD?
     
  3. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    Support for 32X and Sega CD is planned for ZOMG v0.9 and v1.0 (in whichever order they're reimplemented in Gens/GS II). I'm implementing one component at a time in order to make sure everything works correctly without getting overwhelmed with stuff.
     
  4. Sappharad

    Sappharad

    Oldbie
    1,413
    70
    28
    It would be nice if SRAM was an optional common component. It seems that most emulators don't back this up which causes strange effects if loading a savestate in some games, for example the emeralds collected in Retro Remix.

    I would consider this important for a 'perfect' representation of the game's state.
     
  5. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    SRAM is planned to be optional. In Gens/GS, it's currently saved as part of the Gens Rerecording extensions to the GSX save format. The ZOMG format will allow both SRAM and EEPROM to be saved, but will not require it. Gens/GS II will make it a user-selectable option.

    (The Sega CD portion of the ZOMG spec will allow both internal and cartridge BRAM to be saved too, with a user option to disable loading and saving of BRAM from savestates.)

    Also, since ZOMG is a Zip archive, it'll be really simple to extract an SRAM file from a savestate. It'll use the same format as Gens/GS II right now (full even/odd byte dump). I'll probably put it in common/
    instead of MD/, since SMS and Game Gear games also support SRAM.
     
  6. Overlord

    Overlord

    Now playable in Smash Bros Ultimate Moderator
    19,218
    965
    93
    Long-term happiness
    Question - will there be a GSX <===> ZOMG converter?
     
  7. Hivebrain

    Hivebrain

    Administrator
    3,047
    154
    43
    53.4N, 1.5W
    Github
    What about storing a screenshot for thumbnails or something?
     
  8. Overlord

    Overlord

    Now playable in Smash Bros Ultimate Moderator
    19,218
    965
    93
    Long-term happiness
    Oooh, that's a good idea.
     
  9. Nibble

    Nibble

    Oldbie
    If VDP data is going to be stored in the format, a separate screenshot may be unnecessary since it can just render a frame from the savestate data. I might be wrong though.
     
  10. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    I'll probably write an external converter for this. In doing so, I may convert the ZOMG code to a dedicated library with public API and stuff.

    That's an optional part of the specification. A thumbnail can be saved as /preview.png in the Zip file.

    This method wouldn't work because the VDP doesn't store information about raster effects, e.g. the underwater effect in Sonic games. Raster effects are done by changing VDP registers and/or Color RAM halfway through the screen, so we'd end up with the wrong palette for one part of the screen. As an example, Genecyst doesn't properly emulate raster effects, which can be seen by loading any Sonic game and going to a water level. If the water isn't past the top of the screen, it will use the above-ground palette for the water area.

    Also, using a PNG image would make it easier to write a preview plugin for various file browsers. :)
     
  11. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    ZOMG! Changes today:
    • New library LibZomg. This library will encapsulate most of the ZOMG reading/writing code, which will make it easier to write multiple programs that support the ZOMG format (e.g. the GSX to ZOMG converter).
    • Initial preview.png support. A screenshot is currently saved when savestates are saved; however, the preview isn't loaded yet. (LibZomg doesn't support converting raw image data to PNG, so that has to be done in the UI code; however, Zomg::savePreview() will verify that the image data is in PNG format by checking the header, and non-PNG images will be rejected. Likewise, Zomg::loadPreview() will also reject non-PNG screenshots once it's implemented.)
    • Fixed a stupid bug that caused the Z80 emulator to crash sometimes when loading a savestate. Specifically, BC2, DE2, and HL2 were being loaded as BC, DE, and HL.

    Regarding the last one: That seemed to trigger a bug in the Z80 emulator where it would read past the allocated memory. The program counter went over 0xFFFF. This is due to an "optimization" that maps the Z80 PC to an x86 pointer instead of keeping it as an array index. I'm not sure if it could be used as an exploit, but it can definitely crash the emulator.

    It'll probably be fixed in the C rewrite later.

    EDIT: Random question. Right now, in MD/M68K_reg.bin, the address registers (A0-A7) are stored before the data registers (D0-D7). Should I change it so data registers are stored before address registers, or does it not really make that much of a difference? <_<

    EDIT 2: I decided to take a look at how Gens Plus stores its thumbnails. At the end of the savestate file, it saves a 32-bit LE "magic number", 0x00534D47 ("GMS\0"), followed by a raw 16-bit image that's 25% of the size of the original screen (80x60 for MD, 64x48 for SMS). Interestingly, this thumbnail isn't compressed, even if the rest of the savestate is compressed. I suppose it was done this way to make searching for a preview image faster. I may add support for this extension in the GSX to ZOMG converter, but it won't appear as good as the previews taken by Gens/GS II, since the screenshots were reduced to 25% of their original size. (Gens/GS II's preview screenshots are full resolution for non-interlaced mode, and the current field for interlaced. I'll add 448/480-line modes eventually.)
     
  12. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    Latest git code now shows preview images from savestates when selecting a different savestate slot. The image doesn't go away after the timer expires yet, but it will go away if you select a save slot that either doesn't have a savestate or a save slot that has a savestate that doesn't have a preview image.

    Also, MD/TIME_reg.bin is implemented for both save and load. Gens/GS II only uses the SRAM_ctrl field right now. The SSF2 bankswitching fields will be supported later.
     
  13. BiafraRepublic

    BiafraRepublic

    SSR DJ, longtime TSS staffer, and all-around nice/ Oldbie
    29
    0
    0
    Unincorporated Galveston County, TX, USA
    A/V Hijack: Mondays 6-8PM GMT/BST on SSR - http://j.mp/ssrsite
    Instead of a converter, why not store a GSX file inside the ZOMG? After all, since ZOMG is basically a container for the memory, registers, etc., along with a preview image, and the GSX files aren't that big to begin with, it shouldn't be too much trouble to do...
     
  14. Sik

    Sik

    Sik is pronounced as "seek", not as "sick". Tech Member
    6,718
    1
    0
    being an asshole =P
    Well, I suppose the idea is to eventually fade GSX into obscurity, so it may be a better idea if we don't force emulators to support it and use an external tool.

    I generally avoid carrying savestates between emulators anyways. Something is always bound to break...