don't click here

LZ00 compression

Discussion in 'Technical Discussion' started by NickW, May 21, 2008.

  1. NickW

    NickW

    Member
    I need help with another of Sega's compression formats, this time the LZ00 compression format. This one doesn't seem as nice as the LZ01 compression.

    Here is what I know about it:

    0 - 4: 4C5A 3030 (LZ00)
    4 - 8: Compressed file size
    16 - 48: Uncompressed file name
    48 - 0x56: I don't know what this is
    64: Start of data

    I believe this is what the file header should look like for the files, as this is the header uncompressed TEX files have:
    5445 5830 0100 0000 0000 0000 0000 0000 4749 4D00 3000 0000 D093 0000

    Here are some files:
    LZ00 compressed TEX (TEZ)
    Another LZ00 compressed TEX (TEZ)
    Uncompressed TEX file
     
  2. Nemesis

    Nemesis

    Tech Member
    My first thought was deflate compression (zlib), but it's not. Whatever it is, it's a pretty solid compression format. You'd really need an uncompressed version of one of these files to compare against in order to make any sense of it, and even then, it'd be rough. The output of this compression looks very dense; too dense for any basic algorithm I'm familiar with. I wouldn't be surprised at all if it was a more heavy duty format they borrowed from elsewhere. It's not zlib though.

    There are a few gaps I can fill in with the header though:
    0x00 - 0x04: 4C5A 3030 (LZ00)
    0x04 - 0x08: Compressed file size
    0x10 - 0x30: Uncompressed file name
    0x30 - 0x34: Uncompressed file size (probably)
    0x34 - 0x38: File creation time/date (either creation or modification anyway)
    0x40: Start of data
     
  3. NickW

    NickW

    Member
    It's a shame Sega had to use an annoying compression format. The only thing that would probably be the easiest you can do is to work off the smallest LZ00 compressed file, which I believe is 847 bytes:

    http://www.puyonexus.net/files/misc/DOWNLOAD.mrz

    Although it's probably still not good enough.
     
  4. Nemesis

    Nemesis

    Tech Member
    I think it'll be easier hunting around for the library they used. No company builds their own compression formats nowadays. Compression theory is a specialist area of computer science that most programmers know little about. It's not an effective investment of time or money for a company to pay a few programmers to build a library which supports multiple compression methods, when you can find free or very cheap libraries nowadays that are better than anything those guys are going to come up with in 6 months.

    At this point, I think this is most likely the library they're using:
    http://www.oberhumer.com/opensource/lzo

    Actively supported, lots of different algorithms, commercial licensing offered, and it's been used on lots and lots of different platforms, including consoles like the N64, Dreamcast, and the PS1/2.
     
  5. NickW

    NickW

    Member
    I took a look at it and your right, it seems likely that this is the library Sega used.

    One particular thing is that they have things for LZO1 compression. Sega used LZ01 compression in 15th Anniversary.

    I'll take a look tomorrow to make sure.

    EDIT: When I was looking through the ps2 game binaries for Fever 2 & 15th Anniversary' I noticed the string "lzpress" near the file system data.
     
  6. NickW

    NickW

    Member
    I'm going to bump this since I managed to figure out how everything is stored in that ginormous DATA.BIN file (I'm sure there is a topic somewhere here about it.) and nmn isn't able to figure out this compression format. And lo and behold, Sega decided to leave 2 compressed (& encrypted?) and uncompressed versions of the same file.

    http://www.puyonexus.net/files/misc/01_05.MRZ (LZ00 compressed)
    http://www.puyonexus.net/files/misc/01_05.MRG (uncompressed)

    http://www.puyonexus.net/files/misc/02_04.MRZ (LZ00 compressed)
    http://www.puyonexus.net/files/misc/02_04.MRG (uncompressed)