[NeKit]

Most of Sonic Boom (Wii U version) data (except movies and sounds) is contained in .wiiu.stream. Movies are Bink Video and can be easily viewed by BIK Player.

As for .wiiu.stream, QuickBMS script can be used to extract files from them in compressed format. I've written a small Python 3 script to decompress them: https://github.com/N...onic-boom-tools . It may have yet unknown bugs, but so far it's able to decompress things like .dds textures, .chr models (can be viewed by Noesis with cryengine_cgf.dll plugin) and level geometry. Compression itself seems to be LZSS variation. The format is as follows:

Size data
    First byte
        Bit 0-5 size bits
        Bit 6   If set, copy size+3 bytes from unpacked stream - offset, otherwise read size bytes into unpacked stream
        Bit 7   If set, read one more byte and add bits 0-6 from it to the end of size value
    Next bytes
        Bit 0-6 size bits
        Bit 7   If set, read one more byte and add bits 0-6 from it to the end of size value
Offset data (only if Bit 6 of first size byte is set)
    Bit 0-6 offset bits
        Bit 7   If set, read one more byte and add bits 0-6 from it to the end of offset value

For example, C1 BD 60 01 tell us to copy last 1+3 bytes (0xC1 & 0x3F) << (7 * 2) | (0xBD & 0x7F) << 7 | 0x60 = 24288 times (that's it, back offset is constant when copying).

That's all for now. I'm probably going to write another script to extract .wiiu.stream files, since it's more convenient to have it both extracted and decompressed at once, and QuickBMS script seems to fail in some cases. Many thanks to TwilightZoney and Paraxade for buying the disc and getting the files ripped.

[Paraxade]

I wrote a tool for .stream files last week that can be downloaded here. It handles extraction + decompression simultaneously and comes with some bat files to make it simple to extract everything.

also, here's some C++ code for decompression:

u32 getSize(u8 *&src, bool seek)
{
    u32 size;
    u8 byte = *src++;

    if (seek) size = (byte & 0x7F);
    else      size = (byte & 0x3F);

    while (byte & 0x80)
    {
        byte = *src++;
        size = (size << 7) | (byte & 0x7F);
    }

    return size;
}

bool decompress(u8 *src, u32 src_len, u8 *dst, u32 dst_len)
{
    u8 *src_end = src + src_len;
    u8 *dst_end = dst + dst_len;

    while ((src < src_end) && (dst < dst_end))
    {
        u8 byte = *src;
        u32 size = getSize(src, false);

        // repeat data: copy back from decompressed stream
        if (byte & 0x40)
        {
            size += 3;
            u32 seekSize = getSize(src, true);
            u8 *seekStart = dst - seekSize;
            u8 *seekEnd = dst;

            u8 *dstCopy = seekStart;
            for (u32 b = 0; b < size; b++)
            {
                *dst++ = *dstCopy++;
                if (dstCopy >= seekEnd) dstCopy = seekStart;
            }
        }

        // new data: read directly from compressed stream
        else
        {
            memcpy(dst, src, size);
            dst += size;
            src += size;
        }
    }

    if ((src == src_end) && (dst == dst_end)) return true;
    else return false;
}

.stream format is pretty simple:

u32 strm_magic;

// read files until EOF
struct file {
  u32 compressed_size; // if 0, file is not compressed; use decompressed size
  u32 decompressed_size;
  u32 hash; // the game uses this to verify the integrity of the file data
  u32 unknown;
  string filename;
  u8 data[];
};

I'm guessing the reason that QuickBMS script fails sometimes is because it doesn't account for uncompressed files.