don't click here

DOOM 32X IWAD Converter

Discussion in 'Technical Discussion' started by saxman, Dec 15, 2009.

  1. saxman

    saxman

    Oldbie Tech Member
    LATEST VERSION: v1.10

    As an experiment, I wrote a tool to convert the IWAD data from the 32X version of DOOM over to the PC format. The sprites and patches were in a slightly different format, and the IWAD pointers were big-endian, so there was a lot of converting that had to take place.

    http://www.4shared.com/file/198175058/d230...wad32x_110.html

    This tool successfully converts the sprites, floors, patches, and brings over the other data included. Although the wall patches show up just fine, you won't be able to view the data in the TEXTURE1 file properly since the patches that show up link to the main DOOM IWAD file used by XWE (if you use another tool to view the data, this may not even be a problem), but the data is correct. Also, maps haven't been converted, so although the data is there, it's not viewable in any way.

    Maybe with some time, I can convert PC data over to the 32X format so people could change the 32X game. It's just a thought, though the game is very dated, and the 32X version certainly isn't the most popular of versions.

    Anyway, if you're curious to try it out, go ahead. Your ROM or 32X IWAD filename will have to be DOOM.BIN, DOOM.32X, DOOM.WAD, DOOM32X.BIN, DOOM32X.32X, or DOOM32X.WAD. The tool converts everything automatically.



    EDIT: 24 level version of DOOM 32X -- http://forums.sonicretro.org/index.php?s=&...st&p=390142
     
  2. So does this mean I can eventually start porting Chex Quest to the 32X?
     
  3. Glisp

    Glisp

    That one weird guy that does stuff. Member
    1,278
    2
    16
    Bloomington, IN
    None at the moment I'm afraid.
    Perhaps we can add the missing levels in the 32X version by taking them out of the PC version eventually. This is really great Sax!
     
  4. I just looked through the WAD. Wow, this is way more impressive than I expected it was going to be. I notice the weapons are only half the size, is there a reason why they did that?
     
  5. Overlord

    Overlord

    Now playable in Smash Bros Ultimate Moderator
    19,470
    1,083
    93
    Long-term happiness
    Either the fact the PC probably would have had a larger resolution, or to try to ease some pressure off the 32X. It's far from the most efficient port of Doom =P


    Nice work, Sax!
     
  6. saxman

    saxman

    Oldbie Tech Member
    For those interested to know, I have discovered that the data files used by each map (BLOCKMAP, THINGS, LINEDEFS, etc) that were renamed just happen to have a different format from the PC data files. The data file names that weren't changed use the same formatting, but with big-endian data as opposed to little-endian. I don't think any of this is by accidental. I thought this was a bit interesting. I've figured out the format, I just haven't written any code to convert them into the PC format yet. That's coming up next.

    I'd also like to put out there that for some reason when you use the "patches" filter in XWE, it doesn't show all the patches. I have no clue why since they're all clearly between the "P_START" and "P_END" markers. I suppose it's a bug in XWE, because I tried another WAD utility which detected ALL the patches. Anyway, if you want to see all the patches, do NOT filter anything. *sigh* I miss WinTex! =(

    As for the question about the weapon frames, the 32X port uses a screen size of 160x180. Essentially what that means is interlacing is used. That's why the width of the weapon frames are cut in half.

    On a side note: contrary to popular belief, the 32X actually can handle DOOM a little better than most people think, and you can see this proof in one of the later beta versions of the port, but with a 30 fps timing used, everything ran a tad bit on the "fast" side, so the final version of the game uses some timing mechanism to make things run at the equivalent of 35 fps (the rate used in the PC version.) THAT is what made the 32X port so darn choppy. I think with a bit more time, they may have been able to improve it a bit.

    On yet another side note: the 32X version is based on the Jaguar port. The Jaguar DOOM source code actually has C preprocessor stuff in the code to compile the 32X version of the game. I haven't even attempted to compile it, but with this technical discussion of the 32X version, I thought I'd point that out.
     
  7. Conan Kudo

    Conan Kudo

    「真実はいつも一つ!」工藤新一 Member
    478
    1
    18
    Do you plan to release the source of this tool?
     
  8. saxman

    saxman

    Oldbie Tech Member
    I might. I haven't really thought about it. I usually don't like releasing source code to my stuff until it's in a state of completion. This tool could do way more than it does right now.
     
  9. Conan Kudo

    Conan Kudo

    「真実はいつも一つ!」工藤新一 Member
    478
    1
    18
    If anyone is interested in trying that, I found a homebrew devkit for 32X an hour ago, and I figured somebody might find it useful. Also, here is the Jaguar Doom source port.

    Note: I believe the homebrew devkit requires Linux, as a cursory examination into the tarball showed ELF binaries for the compiler and assembler, rather than Mach-O for OSX or PE for Win32.

    NOTE: I'm a moron. It says it right on the page that it is for 32-bit Linux.
     
  10. Chilly Willy

    Chilly Willy

    Just pining for the fjords Tech Member
    755
    21
    18
    Heart of NASCAR
    Doom CD32X Fusion
    First, lemme say I think you're doing great here. :)

    The 68000 and SH2 processors used in the Genesis/32X are big-endian, so it's not surprising that they "flipped" the endianness of the WAD file around where applicable. "Lesser" systems need all the help they can get.


    All the older console ports use low-detail mode to boost the render speed. It's too bad that they don't give you the option of high-detail, but in the case of the 32X, they probably don't because it would use too much ram. That's the biggest issue with the 32X - lack of ram. They probably switched from high-detail to low in the betas when they ran out of memory.


    They should just let it run at 30 Hz. Sure the PC (and other ports) may run at 35 Hz, but it's not like you're going to be doing DM with your 32X against a PC. They don't need to be at the same rate. That's what I do in Wolf32X - it also is supposed to run at 35Hz, but I set the game rate to the vblank rate.


    The issue there is that the 32X specific code isn't included, just the Jaguar specific code. :(

    If they had included the 32X specific code, I'd have already got it compiling. Maybe I should devote a little time to seeing if I can recreate the 32X specific code. :)


    Yeah, that's my toolchain I use for Wolf32X (among other things). The current version is here: gendev20091130.7z

    I've been working on the new menu for the Neo Myth flash cart, and I've got more stuff for the SDK, so I'll probably be putting out an update in a week or two.
     
  11. saxman

    saxman

    Oldbie Tech Member
    New version of my conversion tool -- http://www.4shared.com/file/175540942/bb9d...32X_121709.html

    This version writes the PNAMES file to make textures properly viewable, and it converts all the maps over to the PC format. I looked at them through DeePsea with the R3DEdit add-on, and here are some snapshots I took:

    [​IMG]

    [​IMG]

    [​IMG]



    I was also able to successfully play the maps in R3D. So everything is definitely there and working. Of course, my weapons were half their normal size, but that's okay =)



    Note: you have to delete the COLORMAP file out of the WAD if you wish to view the level properly. The 32X COLORMAP file has some extra information in it that isn't compatible with the PC version.

    Also, you may or may not have known that the Jaguar version supported multiplayer. The 32X maps still have the deathmatch items and starts, as well as a player 2 co-op start. I guess they just didn't worry about them or didn't get around to removing them. Clearly some maps were heavily sized down to make room in the ROM.



    The next step is to port things back to the 32X format.
     
  12. Shadow Hog

    Shadow Hog

    "I'm a superdog!" Member
    So how long until somebody ports Doom 2 maps? Sans all the new monsters/super shotgun, of course (and sadly).

    Or hell, just SLIGE/OBLIGE maps.
     
  13. Conan Kudo

    Conan Kudo

    「真実はいつも一つ!」工藤新一 Member
    478
    1
    18
    Does Doom II use the same engine as Doom?
     
  14. saxman

    saxman

    Oldbie Tech Member
    Exact same engine. In fact, the DOOM2.EXE file will run the original game just fine. If DOOM2.WAD isn't found, then it searches for DOOM.WAD. The same deal applies to Final Doom -- that EXE searches for the two Final Doom WAD files, then the DOOM II WAD, then the original. Everything is backwards compatible.

    I'll see about porting a DOOM II level over to the 32X game (after it's modified to get rid of PC and DOOM II specific stuff.)
     
  15. Conan Kudo

    Conan Kudo

    「真実はいつも一つ!」工藤新一 Member
    478
    1
    18
    That explains why the Doom source ports remained popular for as long as they did. Two Doom games used the same engine, so all the improved versions worked with both games. More incentive to work on the engine.

    The Jaguar codebase isn't the most up to date. Perhaps some of bugs in the Jaguar codebase could be fixed by using code from other engines? Or even bringing the platforms that Jaguar supports to other codebases that are more actively developed?
     
  16. ICEknight

    ICEknight

    Researcher Researcher
    You mean there's actual files inside the 32x ROM?
     
  17. Sintendo

    Sintendo

    Member
    249
    0
    16
    The ROM contains a WAD file (format used by all Doom games), which also contains files.
     
  18. Overlord

    Overlord

    Now playable in Smash Bros Ultimate Moderator
    19,470
    1,083
    93
    Long-term happiness
    Yeah. A WAD is in essence a compressed archive with it's own internal file structure a la ZIPs or RARs. Nintendo DS ROMs actually use a similar system, too.
     
  19. saxman

    saxman

    Oldbie Tech Member
    You can find the WAD file at 0xBB000 in the finalized ROM. It starts with "IWAD", followed by the number of "lumps" (data files in DOOM terminology), and then the address to the file lookup table which contains the size, address, and name of each file. It's funny to think of a single ROM image having files inside, but if you think about it, a single CD image is made up of files, as is anything. It's all in how you think about it and how you define what a file is and isn't.



    And just a quick update, I am part way into coding the PC-2-32X part of my utility. I had to take a break from it to think about how I wanted to do it, because the TEXTURE1 file is ONLY used to identify the name of a specific patch (one only -- the 32X game ignores all other patches), as well as the size of the patch (the size information is taken out of the patches in the 32X version), despite the rest of the information taken from from the PC version still being part of it.

    For those not quite familliar with how the DOOM engine does things, there are wall patches that are used collectively to make up wall textures. There's a texture file, called TEXTURE1 (and also TEXTURE2 for the registered version of the first DOOM game) that is used to create wall textures. You can apply multiple patches, and can locate them anywhere on the wall texture. I suppose this cuts down on space, but the 32X version of DOOM ignores all but one patch for each texture. In fact, the patch who's name matches a texture is used for that texture, regardless of the patch number indicated in the TEXTURE1 file. I guess they didn't feel like reformatting the TEXTURE1 file and just applied some quick and dirty changes to the engine to handle a single patch to make things simplified.

    This complicates things because typically in a DOOM WAD file, patch names do NOT match the texture names using those patches. I decided that the easiest thing to do is ignore the TEXTURE1 file and create a new one based on the patches found in the WAD. Textures in map files will have to be reassigned, but that's a heck of a lot easier than doing what I'm doing right now. Still, you could probably port a map in 10 minutes with my tool and a DOOM map editor. And the good news is the DOOM 32X port doesn't crash when it can't find a texture being used (unlike the PC version.) Several maps use nonexistent textures which are remains from the PC maps.

    Basically the end result of all of this is this: porting maps won't be automatic, but with a little patience, you shouldn't find it difficult to do either. My first task is putting Entryway (MAP01) from DOOM II into the DOOM 32X ROM. I'll have a utility update with snapshots to show it off when I finish. Look for it soon =)
     
  20. Shadow Hog

    Shadow Hog

    "I'm a superdog!" Member
    Kickass.

    Although I do wonder how Doom 32X is going to handle some of the larger/taller stages; I mean, Doom had an arbitrary limit on how tall sectors could be that was removed between the two games (IIRC); that might cause some issues here and there. Then again, it's not like you're porting MAP15 or anything (which is apparently a LOT shorter than I'd thought it was... dang.)