don't click here

Need testers for Gens and Gens/GS ROM compatibility

Discussion in 'Technical Discussion' started by GerbilSoft, Apr 16, 2015.

  1. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    I'm getting close to a technical preview release of Gens/GS II, and one of the things I'd like to identify are ROMs that have issues with Gens and Gens/GS r7 to see if Gens/GS II fixes them, and if not, to indicate what needs to be improved.

    I created a page on Sega Retro: Gens/GS II Compatibility List - this list details a few games and their compatibility on Gens and Gens/GS r7. There's also a column for Gens/GS II, but since there's no precompiled build yet, I'm pretty much the only one who can test it. Anyways, I need help testing various games on these emulators, so if anyone would like to contribute, please test various games and add them to the table.

    Download links:
     
  2. Sappharad

    Sappharad

    Oldbie
    1,413
    70
    28
    This is only slightly related, but I wanted to ask anyway. Do you feel that Gens GS II is still necessary? I figure that you're still working on it because you've already put so much effort into it already and it's basically yours, but I'm curious to know what you feel regarding what role Gens GS II will play in relation to the other Genesis emulators available now. In the years since you started, Nemesis released his thing that's supposed to be extremely accurate and meanwhile Genesis Plus GX added Sega CD and claims to be compatible with all official Genesis and Sega CD games while still being very fast and fairly accurate. GPGX has a good UI with some debugging features via BizHawk's inclusion of it on Windows and OS X, and that's being actively maintained.

    Where do you see Gens GS II sitting in relation to those? What benefits does it have? Last I checked your repo, you weren't re-writing any of the 32x stuff yet so at its current state it seems very similar to GPGX. Just wondering what your current goals are basically.
     
  3. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    The main reason why Gens/GS II has been lagging is because I've been putting it off for a good long time. :v: (Also the AV sync stuff, which is still not quite production-ready but is getting there.)

    I still think it's necessary because unlike most of the other emulators, it's Linux-focused (though it still maintains cross-platform compatibility), whereas everyone else either ignores Linux or considers it as an afterthought. Also, I still want to learn how all of the systems work together, so it's a good way for me to figure out the internals of the hardware.

    That, and it's never a bad idea to have more than one implementation for a given task. (See the web browser market in the early 2000s.)
     
  4. What's the word on Pico compatibility on this release? I may go test it if it's possible it's a step up from Kega Fusion and PicoDrive.
     
  5. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    None right now, unless you count "game loads and shows something". Still focused on MD only right now.

    Pico is basically MD - Z80 - YM2612 + ADPCM chip and a different I/O mapping, so it shouldn't be too hard to implement, but it'll take time. (ADPCM will definitely take longer, since Gens obviously doesn't have any code for it right now.)

    For reference: http://notaz.gp2x.de/docs/picodoc.txt

    Basically, the $Axxxxx range is gone. Instead, I/O is mapped to $800000:
    Code (Text):
    1.  
    2. $800001.b: Version register. Bits 6-5 indicate region (00=JP, 01=EU, 10=US, 11=Asia); other bits are unknown.
    3. $800003.b: Buttons: [PEN x x ACTION RIGHT LEFT DOWN UP] (action is the Red button)
    4. $800005.b: Pen X coordinate, MSB.
    5. $800007.b: Pen X coordinate, LSB.
    6. $800009.b: Pen Y coordinate, MSB.
    7. $80000B.b: Pen Y coordinate, LSB.
    8. $80000D.b: Page register. 00 == closed, 01 == page 1, 03 == page 2, etc. (one bit per "open" page)
    9. $800010.w: PCM data register.
    10. $800012.w: PCM control register.
    11. $800019.b: Write 'S' here.
    12. $80001B.b: Write 'E' here.
    13. $80001D.b: Write 'G' here.
    14. $80001F.b: Write 'A' here.
    15.  
    Pen X coordinate is $03C - $17C.
    Pen Y coordinate is $1FC - $2F7 (tablet) or $2F8 - $3F3 (book).

    I find it interesting that Pico still has TMSS, even though Sega v. Accolade happened years earlier.

    There's also a few more interrupts compared to the MD.

    EDIT 2: I might try quickly hacking together basic Pico support for buttons, tablet, and page register this weekend.
     
  6. Sappharad

    Sappharad

    Oldbie
    1,413
    70
    28
    That makes sense, although I don't entirely agree. Stuff like Dolphin continues to equally support Windows, OS X and Linux because they have developers willing to maintain each platform.

    In my experience, some of these projects are totally open to supporting other platforms like Linux and OS X, they just don't have anyone to do it. Someone who primarily uses Windows isn't going to want to support a platform they don't use. With BizHawk, I've basically been maintaining the OS X port myself for the past 2.75 years because there's nobody else who wants to work on it. I did a Linux build a couple years ago, (before Genesis and other external cores came into it) but since I don't use that OS it didn't make sense for me to spend time on it. (The main project from the portable branch still compiles on Linux thanks to my changes, but a lot of the UI doesn't function properly and nobody's ported any of the external cores.) I've actually been working on a new UI for better cross-platform support, but that's a long ways from being useful at this point. I know in that case, they've been really supportive of cross platform development. They refactored the code last year to completely separate the UI from the emulation side of things, moved common backend (save states, etc.) to it's own module as well. They even added OpenGL and OpenAL to the Windows version to make things easier for me.

    I guess my point is that from my perspective, I don't think the majority of emulator developers are intentionally ignoring Linux.

    Congrats to you for at least keeping this up, and filling in the gap on the Linux side of things. Maybe someday your core will replace GPGX as the one everyone wants to port, if it reaches the point where that would be beneficial.
     
  7. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    Reaching GenPlusGX-level compatibility is going to take quite a while, and not just because of the internal hardware emulation. genplus-gx also emulates a bunch of mappers, including various bootleg and unlicensed mappers, and "T-5740*" mappers for certain recent titles.

    There's also emulation for Game Genie and Action Replay hardware, which I don't think I will be implementing anytime soon. (The only reason to implement them is if you're really nostalgic for the Game Genie or Action Replay menus. :v:/>)

    On a sidenote: Once I get the technical preview out (after getting video sync to an acceptable level), I'm going to start working on VDP Mode 4 emulation. This will be in MD mode, so I'm going to start by porting the Sega Master System v1.3 boot ROM to 68000 (without the cartridge check). That will be fun. :)

    EDIT 2: Well, maybe not porting the boot ROM. 5,000+ lines of mostly-uncommented Z80 assembler. :v:
     
  8. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    Bumping for testers.

    I'm looking for some people to help test Gens/GS II's minimal SDL port. Currently it requires compilation from source, since I don't have an automated build server set up. I'll probably get that done once I finish up the initial release.

    Gitweb location: http://www.dusers.drexel.edu/gitweb/gitweb.cgi/~korth/gens-gs-ii.git/shortlog/refs/heads/gens-sdl-basic-frontend

    Officially supported systems (tier 1):
    • 32-bit Linux (Ubuntu 14.04+; 64-bit will work with 32-bit compatibility packages)
    • 32-bit Windows XP, Vista, 7

    Not officially supported due to lack of hardware, but may work anyway:
    • 32-bit Mac OS X

    Build requirements:

    All systems:
    • nasm-2.x (older versions may fail in unexpected ways)
    • cmake-2.8+ (3.x isn't tested but should work)
    • SDL-1.2 (will be changed to 2.0 before release)

    Linux: (optional; internal versions of these libraries are included, but system versions are preferred)
    • gcc (tested on 4.9+; 4.4 or higher should work, not sure about older versions)
    • zlib-1.2 or later
    • libpng-1.2 or later
    • GLEW 1.x

    Windows: (optional; internal versions of the above libraries are always used)
    • MinGW-w64 4.x (older versions may work, but not as well)
    • Alternatively: MSVC 2010 (newer versions may work; older versions definitely won't)

    After the initial release, I'm also planning on switching to libarchive for decompression. Not only will this eliminate the need for UnRAR.dll on Windows (and /usr/bin/unrar on Linux), it will also add support for .xz, .tar, and other archive formats.
     
  9. Scarred Sun

    Scarred Sun

    Be who you needed when you were younger Administrator
    7,745
    127
    101
    Tower 8 ️
    Welp, this.
    I'm happy to test, but my main area of concern, like Kiddo, is around Pico support.
     
  10. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    Today's update: The SDL port is now using SDL 2. Had to do some conversions for the new SDL 2 scancodes to the Gens keycodes, which are based on SDL 1.2. (May end up just redoing the Gens keycodes based on SDL 2, but that will break existing configurations. Not a huge issue...)

    Among other things, SDL 2 provides:
    • The "software" renderer is actually implemented using OpenGL or Direct3D, depending on platform. The main renderer will still be the custom OpenGL renderer, so this isn't a big deal.
    • The window can now be resized in OpenGL (and Software) mode without flickering.
    • Windowed fullscreen is now supported. Main advantage of windowed fullscreen compared to "regular" fullscreen is it doesn't change the display resolution, so things don't get randomly shifted around.

    Tomorrow I'll look into doing a quick hack for basic Sega Pico support. This will consist of disabling the Z80 and YM2612, removing the $A00000 I/O area, and adding the custom $800000 I/O area, plus using the mouse to emulate the tablet. Note that anything depending on ADPCM interrupts won't work until ADPCM is implemented.

    EDIT: [2015/07/12 12:10 AM EDT] New branch gens-sdl-pico based on gens-sdl-basic-frontend. It has a new class EmuPico, based on EmuMD, that's currently nearly identical except it doesn't use the Z80 or YM2612. I'll add support for the $800000 I/O area tomorrow. This branch will be merged back into gens-sdl-basic-frontend before the Tech Preview is released.
     
  11. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    Some random notes:

    • The technical preview will not show the Storyware cartridge onscreen. It'll only show the VDP output. Once I get the regular UI working, I'm thinking of making it look somewhat like a DS emulator: in a 640x480 window, the "bottom screen" will be the tablet, and will show the 320x240 display. The "top screen" will be the Pico Storyware cartridge. Assuming high-res scans, increasing the window size will improve the quality of the Storyware viewport, and the video rendering will simply scale based on the rendering settings.
    • I'll add initial pen support tonight. The pen will be controlled by the mouse. Left-click = pen-click; right-click = switch the logical viewport from tablet to Storyware and vice-versa. A message will appear onscreen indicating the change.
    • Basic support for SMD-format ROMs has been added (gens-sdl-basic-frontend branch), because I can.
    Sometime this week or next week, I'm going to rebuild my Windows toolchain and look into getting an automated build server running, which will allow everyone to test Gens/GS II without compiling it themselves.
     
  12. LocalH

    LocalH

    roxoring your soxors Tech Member
    After you do bugtesting and get your initial release out, wanna work in some PYL MD? :v:
     
  13. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    I've completely forgotten about that. Maybe I could do some stuff on it. (Getting initial tiles for the squares and shuffling working would be a nice start.)

    Anyways, a status update:
    • The Qt menu system has been completely rewritten to use Qt Designer. This makes it easier to maintain. This won't have any effect on the Tech Preview, which is SDL only, but it was something that was bugging me.
    • Screenshots are now saved with a fairly extensive set of metadata. (More on that in a bit.)
    • Initial Pico support has been merged back into gens-sdl-basic-frontend. Pen support is still missing; it will be added in before release.
    • D-pad constraints have been reimplemented. When enabled, this prevents U+D or L+R from being pressed at the same time. Some games get confused by this. In the Gens/GS II implementation, if e.g. U is held down and then D is pressed, U will remain held. D will only be "pressed" once U is released. Gens' implementation always had U and L beat out D and R. Note that this does not affect the Pico controller, since Pico has four discrete buttons instead of a D-pad.
    • Fixed a timing issue on Windows that caused the framerate to be reported as 50-56 fps while the actual game ran too fast. This was caused by a rounding error with QueryPerformanceCounter; the frequency was divided by 1,000,000 before dividing the counter in an attempt to prevent overflow. Since it's using integer arithmetic, this caused significant rounding errors on most machines. (My XP VM has a 3.58 MHz frequency; a Windows 7 PC I was using for testing has a 3.25 MHz frequency.) This issue didn't occur on Wine because Wine has a hard-coded frequency of 10,000,000.
    • Fixed various errors found by cppcheck.
    For PNG metadata, I've added the following fields (with an example from a screenshot I just took locally):
    • sBIT: Significant bits per sample: R=8, G=8, B=8
    • pHYs: Pixel size: 4x4 per unit
    • tIME: Last modification time: 25 Jul 2015, 00:34:08 UTC
    • tEXt: Text data: "Description"
    • tEXt: Text data: "Creation Time" = "2015:07:24 20:34:08"
    The "pHYs" field indicates the aspect ratio. For H40 (320px) mode, this is 4x4. For H32 (256px) mode, this is 5x4, since the pixels should be rendered wider. Note that most image viewers simply ignore this field, so don't expect H32 images to be stretched to the correct aspect ratio when viewing them.

    The "Creation Time" chunk is used for Windows 7, which shows it in Windows Explorer. Note that it's in local time, whereas the tIME chunk is UTC.

    The "Description" field has much more data. (I might change it to "Comments", since Windows XP doesn't show "Description" but appears to show "Comments".)
    Code (Text):
    1.  
    2. Emulator: Gens/GS II
    3. Version: 0.0.0 (git: gens-sdl-basic-frontend/fb58bc07)
    4. OS: Linux 4.1.2-gs_laptop
    5. System: MD
    6. ROM: Sonic 2 (Rev.01)
    7. ROM CRC32: 7B905383
    8.  
    Currently, there's no way to configure what fields are set, but the Qt UI will add some options. Among other things, there's code to save the system username (and/or display name), but this is disabled by default. I will probably also add an option for a "custom" username instead of using the system username.

    A good chunk of the metadata is there for debugging purposes. For example, if someone posts a screenshot of a game not working properly, but has no idea how to determine what revision of a ROM they have, the metadata will indicate the CRC32, which can then be looked up.
     
  14. nineko

    nineko

    I am the Holy Cat Tech Member
    6,298
    475
    63
    italy
    You know the png metadata will be stripped if pngout is used, right? It can be preserved with certain parameters, but...

    If it's not too late, I think you should make the metadata addition an option, even enabled by default, but disableable.
     
  15. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    PNG metadata will be configurable once I get the main Qt UI working. And yes, I know that most optimization tools will remove it. The vast majority of users don't use them.

    Sidenote: gens-sdl now runs on Win98SE, with a few caveats:
    • Must be compiled with MinGW-w64. MSVC 2010 can't target anything older than Win2000, and 2010's the minimum version that gens-sdl builds with.
    • KernelEx is required if using gcc-4.8+, since libstdc++ uses GetModuleHandleExW(). (This could help with MSVC builds, but I haven't tested it.)
    • SDL2 must be compiled with Unicode disabled, and uses of AttachConsole() must be commented out.
    I'm probably not going to distribute a 9x-compatible build, but the option is there.
     
  16. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    Today's update: I've done a ton of changes in the past month or so, though most of them have been low-level and not directly related to the main blockers. (Mostly due to boredom.) Among other things, the Paused Effect is now MMX and SSE2-optimized, and the Fast Blur effect is SSE2-optimized for 32-bit color. gens-sdl also has an ARB fragment program version of the Fast Blur effect, ported from gens-qt4. Both of these effects also have test suites to ensure they work correctly.

    There's also MMX- and SSE2-optimized byteswapping code, which is used quite a bit when loading ROMs and loading/saving savestates.

    Other changes include some Win32 compatibility improvements (use of "\\?\" for long filenames on NT-based systems) and the use of popt for command line parsing.

    The major blockers right now other than packaging:
    • Sega Pico pen support.
    • Fix a DMA regression caused by attempting to fix test failures in VDPFIFOTesting. The DMA subsystem needs a complete rewrite, which isn't doable at this point, so I just want to get it working back to how it was in Gens/GS r7.
    I'm planning on releasing Gens/GS II Technical Preview 1 sometime this month. Hopefully I can get it all done.
     
  17. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    zomg, an update.

    A minor shadow/highlight bug was pointed out to me by flamewing and Jorge. In a nutshell: the shadow/highlight sprite operators (palette colors $3E and $3F) don't just mask the sprite they're in; they mask all lower-priority sprites on the same pixel. (Not 100% positive about higher-priority sprites, but I assume those take precedence.)

    Left = before, right = after. (Confirmed on Kega, Regen, and hardware.)

    [​IMG] [​IMG]

    [​IMG] [​IMG]

    I'll probably add an automated test case for this later to ensure that it doesn't accidentally get reverted.
     
  18. flamewing

    flamewing

    Emerald Hunter Tech Member
    1,161
    65
    28
    France
    Sonic Classic Heroes; Sonic 2 Special Stage Editor; Sonic 3&K Heroes (on hold)
    High priority actually makes no difference; the only thing that actually matters is which sprite gets drawn first. If the sprite with the operator is drawn before, it masks sprites that come latter; if a sprite gets drawn before the operator, it masks the operator instead.

    The ROMs I sent you actually tests this too; so matching the image from Kega/Regen/Real hardware means it is correctly implemented.
     
  19. GerbilSoft

    GerbilSoft

    RickRotate'd. Administrator
    2,971
    76
    28
    USA
    rom-properties
    That's what I meant by high priority. Link priority, not plane priority. (It's a bit confusing because the same term refers to different things. :v:)

    I was wondering whether or not the test ROM handled that, so that's a good thing. (I compared it to Kega Fusion 3.63x and the "after" images matched.)
     
  20. flamewing

    flamewing

    Emerald Hunter Tech Member
    1,161
    65
    28
    France
    Sonic Classic Heroes; Sonic 2 Special Stage Editor; Sonic 3&K Heroes (on hold)
    For future reference, the images generated by the ROMs go like this:
    Row 1: no operator sprites, alternating low and high priority, each pair of sprites come from a different palette line. The purpose of this is to test abnormal rendering of colors in sprites on Shadow/Highlight mode. Basically, confirmation that only color 14 is abnormal. The last two sprites use palette line 3, hence they have 2 operator blocks each.

    Rows 2 and 3: as above, but all non-operator sprites are covered by low priority shadow operators that come before them in link order. Split in two lines due to sprite pixels per line limit.

    Rows 4 and 5: as above, but shadow operators are high priority.

    Rows 6-9: as rows 2-5, but with highlight operator instead.

    Row 10: Alternating priority on both highlight operators and sprites, sprites come before operators in link order. Only 4 of them because of 80 sprites total limit.

    Now that I think of it, the last row is actually bugged and the alternating high and low priorities should behave differently for sprites and operators. Oh, well...