Sonic and Sega Retro Message Board: [Up: 10/16/04] TSDC V2.2, KENS V1.4 + Source - Sonic and Sega Retro Message Board

Jump to content

Hey there, Guest!  (Log In · Register) Help
  • 2 Pages +
  • 1
  • 2
    Locked
    Locked Forum

[Up: 10/16/04] TSDC V2.2, KENS V1.4 + Source Release Candidate 3 - Bug fixes

#1 User is offline Magus 

  Posted 26 August 2004 - 04:32 PM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
-----------------------------------------------------------------

LATEST UPDATES

KENS V1.4 RC3
* Corrected a bug that prevented the functions that output data to a buffer to create the buffer (could only use already initialized buffers). Thanks a LOT to Korama for helping me fixing this.
* Fixed all the compression functions so they produce files with an even size
* Included the Delphi unit corrected by Korama (my changes to this unit were not really good - I'm not a Delphi programmer)
* Headers and units are now in a sub-directory named "Include"
* Updated the info in the README.HTML file

KENS V1.4 RC2
* Turned all functions to local functions, so they all use temporary files
* Removed the temporary local Kosinski functions (as the original functions are now local)
* Added functions that output data to a buffer rather than to a file (thanks to Stealth for asking for that improvement)
* Edited the headers and units to reflect the changes
* Converted README.TXT, VERSION.TXT and LICENSE.TXT to HTML, because it's easier to read
* Updated the info in the README.HTML file

KENS V1.4 RC1
* Corrected a bug in the Enigma Decompression algorithm. Thanks to Stealth for reporting and Nemesis for helping in fixing it.
* Added "Local" Kosinski functions, which copy the file to the Windows temporary directory before compressing or decompressing. This fixes a speed issue when the source or destination files are on a remote computer. Thanks to Stealth for finding out this issue (we spent hours trying to find out why it was so slow) and helping in fixing it (although I used a different method from the one used in the unofficial version that he released).
* Edited the headers and units to reflect the changes in the Kosinski functions

TSDC V2.2 RC3
- Repacked with version 1.4 RC3 of the KENS DLLs

-----------------------------------------------------------------

Hello my friends !

Here's the last release of The Sega Data Compressor. It fully supports both compression and decompression of Kosinski, Engima, Nemesis and Saxman formats. TSDC should no longer be updated (but could be repacked with newer versions of the DLLs), unless there's a new compression format.

As for KENS, it is a pack that contains 4 DLLs, each of them featuring compression and decompression of a format. They are distributed under LGPL, so you have access to their source code. These DLLs can be used by both standard (AKA C/C++ :D), Visual Basic and Game Maker applications. Using these DLLs instead of having the code written in the application allows to correct bugs in the algorithm without having to recompile the application. I made these DLLs because I had several applications using the algorithms, and it was a pain to update all of them each time I changed something in an algorithm. Though, the very first time I compiled them, it was for Beta, but I then realized they would be extremely useful for everybody, me included. TSDC uses these DLLs.

If you need help reusing these DLLs in your applications, don't hesitate to ask me for help.

TSDC-V2.2-RC3 (ACE - 91 KB)
TSDC-V2.2-RC3 (SFX ACE - 153 KB)
TSDC-V2.2-RC3 (ZIP - 228 KB)

KENS-V1.4-RC3 (ACE - 71 KB)
KENS-V1.4-RC3 (ZIP - 125 KB)

KENS-V1.4-RC3-Source (ACE - 43 KB)
KENS-V1.4-RC3-Source (ZIP - 73 KB)

Also available on SourceForge (KENS 1.4 RC3 and TSDC 2.2 RC3 not uploaded yet).

As I heard that some people were asking how that tool works, here are some basic explanations:

- In order to decompress a file, open one of the decompression tool, and choose the ROM from which you want to extract data, using the first "Set" button. Then, choose a file that will receive the decompressed data, using the second "Set" button. Then, you can either choose a location from the list or typing one yourself (in that case, you MUST write it in hex, and without the 0x prefix). Finally, press the "Decompress" button. The status line should say "Processing..." for a very short time (usually 2 or 3 ms, unless it's a big archive), and then "Done !".

- Now, compression: run one of the compression tools, and choose the file containing the data which you want to compress, using the first "Set" button. Then, choose a file that will receive the compressed data, using the second "Set" button. All you have to do now is clicking the "Compress" button. The status line should say "Processing..." for a few seconds, and then "Done !".

- Specific options:
* Enigma: when compressing or decompressing, you can choose to add and remove padding. This option should only be used when compressing or decompressing a S1 Special Stage. If you added padding when decompressing the Special Stage, then you will have to remove it when recompressing.
* Kosinski: when compressing or decompressing, you can choose whether the archive is moduled or not. Be very careful when using this. Moduled archives can't be used in place of standard archive and vice versa. In the decompressor, this option is semi-automatic: whenever you click a location line ending by (M), the checkbox will automatically be checked. Do remember to recompress these archives using the moduled option.
* Kosinski (2): when compressing, you can change the sliding window and maximum recurrence length. This does not have any real use for end-users, but it might be interresting for hackers. If you do not know what I'm talking about, then you will not need these options.

- Warning: Enigma and Nemesis compression formats are data-type specific. You CAN'T use the Enigma and Nemesis compression / decompression tools for archiving. You could use the Kosinski tools however, but this is pointless, as the compression process is quite long, and the compression ratio not interresting enough. As for the Saxman compression format, it can only compress very small files, and the maximum size of a compressed file is 64 K (I think).

- Difficulties downloading: if you can't download the files, try opening the FTP folder instead of clicking that link, then drag-drop the file you want to download from your IE window to the desktop or on any folder.
This post has been edited by Magus: 16 October 2004 - 08:33 AM

#2 User is offline Magus 

Posted 29 August 2004 - 05:16 PM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
Okay, I've worked on it three more days, and made several improvements. The new key feature is the ability to output data to buffers instead of a new file. Enjoy it !

#3 User is offline Korama 

Posted 29 August 2004 - 08:21 PM

  • Posts: 272
  • Joined: 18-May 04
Good job, as usual. :D

I see that you also updated the Delphi unit, nice. :(
However, I don't think that your CompToBuf/DecompToBuf work, at least not the way they are declared in Delphi.

The parameter "DstBuffer" is passed by value. That's fine as long as the caller initialized the buffer, because the DLL will copy the data to the specified address. However, if DstBuffer is NULL, the DLL will change its value (by allocating memory), but that changed value will not be returned to the caller! That would only work if DstBuffer was passed by reference, as pointer to the pointer to a char. ;)
That's "char **DstBuffer", I think.

In Delphi declarations you use "var" to pass parameters by reference instead of value (or "out", in case the parameter is used for output only).

Example:
function NCompToBuf(SrcFile: PChar; var DstBuffer: PByte): Longint; cdecl;

That would pass the address of the DstBuffer variable, not its value. Note that I also changed the type from "PChar" to "PByte", because in Delphi PChar is normally only used for real characters, not for binary data. Another option would be to use the basic typeless "Pointer".
This post has been edited by Korama: 29 August 2004 - 08:28 PM

#4 User is offline Magus 

Posted 31 August 2004 - 03:05 PM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
Thanks for the info. Well, I'm not a Delphi programmer at all, so, could you please correct the unit and test it to check if it works correctly ?

#5 User is offline Korama 

Posted 31 August 2004 - 06:06 PM

  • Posts: 272
  • Joined: 18-May 04
I'll gladly correct the Delphi unit.
But first you should fix your C++ sources, because as I said, a problem will occur if you let one of the CompToBuf functions allocate the buffer for you.

Anyway, let's better continue talking about that in PMs or on MSN. :(

#6 User is offline Magus 

Posted 01 September 2004 - 02:26 PM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
Yep, I replyed to your PM. I'll be fixing them right now. Thanks a lot.

#7 User is offline LOst 

Posted 01 September 2004 - 09:15 PM

  • Posts: 4885
  • Joined: 10-January 03
  • Gender:Male
  • Wiki edits:2
I'm using your data compressor for the first time. However, compressing in Enigma will often end in a filesize not dividable by 2. That will end up in Address Error for any data inserted after the data generated by your program.

You must fix that. The Genesis can't handle data starting an odd address. Applies to all compression format! Not just Enigma.

#8 User is offline Magus 

Posted 02 September 2004 - 05:16 AM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
I'll consider you just asked "Could you please fix this ?". :D

That'll be done for the next release.

#9 User is offline LOst 

Posted 02 September 2004 - 09:45 AM

  • Posts: 4885
  • Joined: 10-January 03
  • Gender:Male
  • Wiki edits:2

Magus, on Sep 2 2004, 01:16 PM, said:

I'll consider you just asked "Could you please fix this ?". :D

That'll be done for the next release.

Thanks!

It's very simple. You just check the output size just before you close the file, and if it is odd, then write one more byte:

if ((output_size & 1) == 1)
fwrite (&zero_byte, 1, 1, file_pointer);

This post has been edited by LOst: 02 September 2004 - 09:45 AM

#10 User is offline Magus 

Posted 02 September 2004 - 12:00 PM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
Yep, that's pretty much the way I wanted to do it in my algorithms.

However, the algorithms provided by Nemesis do not use the ANSI-C I/O functions, so I'll have to be tricky. :D

#11 User is offline LOst 

Posted 02 September 2004 - 04:28 PM

  • Posts: 4885
  • Joined: 10-January 03
  • Gender:Male
  • Wiki edits:2

Magus, on Sep 2 2004, 08:00 PM, said:

Yep, that's pretty much the way I wanted to do it in my algorithms.

However, the algorithms provided by Nemesis do not use the ANSI-C I/O functions, so I'll have to be tricky. :D

I know. And even worse. I tried to port Nemesis' compressor into Watcom C for DOS for use with LOst Library, but something in that code is crashing... Have you experience that? The decompressor is fine! But the compressor is older code, and it is unstable

#12 User is offline Magus 

Posted 02 September 2004 - 04:54 PM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
Are you talking about Nemesis' compressor ? Well, I made a DOS port some time ago, and it worked well if I remember correctly. This was under DJGPP.

Anyway, I'm studying this format in order to make my own algorithm, so I can fix bugs more easily. Let me tell you this is a real pain. But I will end up understanding this format. Even if I have to stay awake 24/24, 7/7. :D

#13 User is offline LOst 

Posted 02 September 2004 - 05:06 PM

  • Posts: 4885
  • Joined: 10-January 03
  • Gender:Male
  • Wiki edits:2

Magus, on Sep 3 2004, 12:54 AM, said:

Are you talking about Nemesis' compressor ? Well, I made a DOS port some time ago, and it worked well if I remember correctly. This was under DJGPP.

No! I mean the Decompressor!

#14 User is offline Magus 

Posted 02 September 2004 - 05:40 PM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
Well, you said compressor earlier. Anyway I was asking if it was the Nemesis or Enigma ones. Seems you're talking about the Nemesis ones. :D However, both the compressor and decompressor worked. No worries though, I'll be making new algorithms for these formats. It might take some time, but I'll do it anyway.

#15 User is offline Magus 

Posted 16 October 2004 - 08:37 AM

  • Fallen Hacker From The Past
  • Posts: 157
  • Joined: 14-April 04
  • Gender:Male
  • Location:Paris, France
  • Project:Translating Grandia 2 PC to French
Hello there !

KENS 2.2 RC3 has been uploaded and is ready for download !
It mainly includes fixes. Check the original post for more info.
As always, feedback is appreciated. ;)

  • 2 Pages +
  • 1
  • 2
    Locked
    Locked Forum

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users