Ecco the Dolphin Reverse Engineering and Preservation Project

Discussion in 'Engineering & Reverse Engineering' started by Arkonviox, Jan 18, 2021.

  1. Arkonviox

    Arkonviox

    Member
    8
    9
    3
    Hi!

    This is my first real post here so I might as well introduce myself. I am Arkonviox, 17 years ago I ran an Ecco the Dolphin website called Arkonviox.com (https://web.archive.org/web/20021001092301/http://arkonviox.com/index.php)

    A year ago I started a project to create an Ecco the Dolphin fan game from scratch. A majority of this project depended on reverse engineering all the graphical assets from the games and anything else I can cannibalize and use toward my new engine. I began this project with no experience with game hacking or 68000 assembly but as of last September I managed to extract most of the graphical assets from all the 16-bit games, including Ecco PC! This includes sprites, level maps and non graphical stuff like object formats and various compression algorithms.

    During that period I also managed to create a new Ecco the Dolphin community based around my new website Ecco the Dolphin Online https://eccothedolphin.online
    The goal of Ecco Online has been to gather all materials from all the games in the series as well as information and document as much as possible.

    Here is a little taste:

    vortex-drone.png

    and for those who are interested in our sprites and level maps:
    Sprite Sheets: https://eccothedolphin.online/sprite-sheets/
    Level Maps: https://eccothedolphin.online/level-maps/

    ElxiC8mVcAUBhOS.png

    At the moment my focus has been to clean up the extraction tools and make them available to the public with tutorials on how to use them. The tool will actually be split from the engine itself and made into a library that I plan to call Black Omen. For those interested in the extraction tool in it's current state, I post the latest snap shots on our discord server.

    Learning 68000 and trying to understand how to reverse engineer these assets was the hardest thing I ever had to learn but I succeeded. The extraction tools can read Mage Drive bitmaps, and partially emulate the VDP to display a variety of graphics. I also had to figure out the formats used in the Ecco games for sprites, and level maps (Each Ecco game has it's own format, except for Ecco Jr, which just uses Ecco 2's format).

    As for Ecco the Dolphin Online's preservation project, we have members of our community who've been working hard to translate materials and restore scanned media, including recreating new versions of box art and various other things.

    Here is some info about our media restoration project:
    https://eccothedolphin.online/media-restoration/

    ElxhyeZUYAAnEJF.jpg ElxhxqHVcAAoNu4.jpg

    I also built an online animated Ecco the Dolphin Text generator that uses code from the actual game to produce different effects.
    https://eccothedolphin.online/ecco-text-generator/

    [​IMG]


    I'm apologize in advance for the overload of information and links, I been meaning to post here for a long time and haven't had the chance. I just basically wanted to introduce everyone to our new community and the insane work we been trying to do. I started this insane project when I seen no one was really covering the Ecco the Dolphin games anymore. Over the years alot of information had been lost, and alot not covered at all. It's been tough but I do have alot of never before seen content that I hope to share over time, both from my reverse engineering efforts and from my insane collection. If anyone here are fans of Ecco the Dolphin please check out our site, and if interested feel free to get involved! We do need all the help we can get and would be happy to have you there! (Our community mostly operates through our discord: https://eccothedolphin.online/join-our-community-on-discord/)

    Anyhow take care and if anyone has any questions feel free to ask!
    Thank you!
     
  2. Asagoth

    Asagoth

    Behold! The mighty, the flawless, salted cod eater Member
    340
    26
    28
    Portugal
    wiki stuff... and a beer... or two... or more...
  3. Arkonviox

    Arkonviox

    Member
    8
    9
    3
    I actually have all that stuff in mostly mint condition and will be scanning it, restoring it and uploading it soon. I actually have alot of stuff that you guys and many others don't have, including cleaner scans. You are more than welcome to them as long as you credit our website. I wouldn't even ask for credit as the purpose of this project is to get everything on the net and preserved (so it doesn't get lost). However, we need to get our name out there (aggressively) so we can keep growing our community and gain contributors who can help with the over all process of getting everything online (I can't do everything by myself). This includes restorations (cleaning up scans), refining (a few folks in our community know how to remove halftoning and more), and remastered media (basically the process of remaking things like box art). We still have a very long ways to go and not everything I have is scanned. But I have ALMOST every ecco item in existence minus a few Tec Toy items. If we can get contributors to get us what we don't have out of that line up (and we need good scans in png [loss less] quality!) that would help.As you can see we aren't playing around :)

    Here is a page from our french manual. It's still not done, as we also will be removing the staples from the final. BTW I had to convert this to jpg just to upload it, but normally we only provide them as PNGs of the highest quality.

    ecco-the-dolphin-french-manual-cover-raw_clean.jpg

    BTW I just want to add, I love what you guys are doing both here and at SegaRetro. If it weren't for the resources from these sites I wouldn't have had the material I needed to reverse engineer these games.
     
    Last edited by a moderator: Jan 19, 2021
  4. Asagoth

    Asagoth

    Behold! The mighty, the flawless, salted cod eater Member
    340
    26
    28
    Portugal
    wiki stuff... and a beer... or two... or more...
    Wow!!! awesome quality :) ... don't worry ... we work as a "bank"... our goal is to preserve stuff ... we don't want to compete with anyone... we will give you credit ... thanks ... you guys are doing great too ...
     
  5. Arkonviox

    Arkonviox

    Member
    8
    9
    3
    Sounds like I came to the right place. If I kick the bucket tomorrow I want to ensure everything gets preserved. We've had too many irresponsible folks in the past who'd hoard and water mark content then disappeared tomorrow along with what is basically historical stuff. I hope to make all that obsolete.
     
  6. Asagoth

    Asagoth

    Behold! The mighty, the flawless, salted cod eater Member
    340
    26
    28
    Portugal
    wiki stuff... and a beer... or two... or more...
    There's a lot of rare stuff watermarked... which is bad... I mean ... I understand that, but ... is just a picture of something ... so why people has to put watermarks in everything?
     
  7. doc eggfan

    doc eggfan

    Are you pondering what I'm pondering? Wiki Sysop
    9,396
    82
    28
    ACT
    GreatMegaLD, GreatSC3k, Great SG1k
    I once had a thought about hacking Ecco as the base for a 2D NiGHTS game on Mega Drive, but it never progressed beyond the idea. I thought about just starting with replacing the Ecco sprite with a NiGHTS sprite and calling it "NiGHTS into Ecco's Dreams"
     
  8. Arkonviox

    Arkonviox

    Member
    8
    9
    3
    I think the biggest issue was these older websites didn't want their content stolen and wanted to keep it exclusive. I don't think they thought of the possibility that their website may one day no longer exist and others would want to keep that content in circulation. That and the amount of work to get this stuff together....But I think it's dumb, it really offers no value to the world at large.

    I should add, because of Sega Retro I was able to go out and obtain items that I didn't know existed, and I'd be happy to help get some of that on this website myself as well. Right now alot of our scans are on our discord, some of it is cleaned up some of it is waiting to be cleaned. I would have had more done but my focus has primarily been on reverse engineering and documenting the games. All of that info I'd be happy to share here as well, it's just getting it organized is alot of work, but I plan to write up an article covering what I found. Also my extraction tool is open source, but not very usable in it's current state as its mostly library code and not an actual tool. I will be documenting that and providing a tutorial for that as well.

    But I'm excited to work with you guys and share it! The Ecco community is starting to come to life and is sorta experiencing a renaissance as I been calling it. So alot of exciting stuff is in the works, with myself and just members of the overall community.

    That wouldn't be hard to do, the rom can be expanded to be larger, and the code that assembles the dolphin sprite tiles would just need to be tweaked. I'll likely be back here again once I have time to document it, I should probably make that stuff a priority to document so I will. I'd love to see all the exciting projects that come from it.
     
    Last edited by a moderator: Jan 19, 2021
  9. Asagoth

    Asagoth

    Behold! The mighty, the flawless, salted cod eater Member
    340
    26
    28
    Portugal
    wiki stuff... and a beer... or two... or more...
    It's true... It looks that the guys from "Caverns of Hope" will resurrect the website...

    here's a rare image of the French adoption certificate ... from issue #13 of the French magazine Mega Force :

    [​IMG]
     
  10. Arkonviox

    Arkonviox

    Member
    8
    9
    3
    Yup we just translated that very same article to English and cleaned up alot of material from it.
    Since that particular part of the article (the certificate) was in such poor shape we didn't use it and opted instead to use the certificate from
    the Ecco the Dolphin Boxed Set. In fact the image you have there is just a blown up version from that MEGA Force article.

    Here is the translated article: https://eccothedolphin.online/le-project-pelagos/

    The original person who created Caverns of Hope quit and shut down the site in 2018.
    The folks incharge now are new, with the main one being a former admin who was appointed I believe in 2014.
    I actually offered to help them out with content but they turned me down due to politics that occurred 16 years ago between myself
    and the sites creator that we later resolved.

    I do wish them luck as we raised the bar this year with our own work,
    as well as becoming the largest community dedicated to Ecco that has ever existed. It was alot of hard work from not
    just myself but the talented folks who help me out, and I'm honored they have invested in this overall project of ours.
    In one year we made a huge difference and introduced some innovative stuff, and this year we plan to set the bar higher! :)
     
    Last edited: Jan 19, 2021
  11. Asagoth

    Asagoth

    Behold! The mighty, the flawless, salted cod eater Member
    340
    26
    28
    Portugal
    wiki stuff... and a beer... or two... or more...
    Now I remember :) ... Arkonviox ... I knew I'd seen that name before... I wish you good luck too guys ;) ...
     
  12. JaxTH

    JaxTH

    Pudding Deity Oldbie
    9,049
    120
    43
    Los Angeles
    Jack shit.
    Making watermarked stuff obsolete is our entire MO. :V

    P.S. You could also mirror any technical findings on a page like this: Ecco the Dolphin/Technical information if you wanted (page doesn't exist yet). This page could be used as a guide: Ristar/Technical information. You could do the same for the other games as well. Sega Retro is currently in read only at the moment, but something to think about.

    Anyway, welcome!
     
  13. Black Squirrel

    Black Squirrel

    this is what KLF is about Wiki Sysop
    Nicely done

    Though I think I'd prefer "non-restored" scans. I respect that it's a massive job and people generally have more of a real-world use for clean images, but when you're in our line of work, the idea is to document "what's actually there" rather than "what you think should be there". Let others edit things as they see fit.

    I do like the maps though, and any unused sprites (watermarks aside). Lots of useful things for Sega Retro when the site becomes editable again :)
     
  14. Arkonviox

    Arkonviox

    Member
    8
    9
    3
    I'll keep that in mind, that gives me something to work with so I can at least figure out how to present it/lay it out.

    I totally understand that, we actually offer three tiers, restored, refined and remastered which are completely remake versions of the material for folks to print and use.
    The restored are mainly to remove dirt, dust and particles, as well as staples, and in some cases to combine two pages together. We never offer refined material without a restored version as a refined version completely changes the material to smooth it out in most cases

    I have all the raw scans but I don't know if we'll be offering those, in certain circumstances we will when restoring them isn't possible.
    Some things don't make a lot of sense to keep as a raw scan, like posters which require multiple scans, but I have all the physical material archived, so if you guys need anything let me know, but since this project is still in it's infancy I'll make sure to keep raw copies on hand if you need them.

    The watermarks on those maps are unobtrusive, each of those maps I added 10 pixels and put the url to our website there, you can remove them by just removing 10 pixels from the height. I hate watermarks but I was forced to put them on there because I started uploading them to other websites and I really need people to find us. We get free advertising from it and it doesn't effect the maps since you can remove the extra 10 pixels from the height I added. The sprites come in two flavors, sheeted and non sheeted, the non sheeted are in zip files and are direct dumps from the game using my software.

    Back to the maps, eventually everyone will be able to dump their own maps! The tool is available and works but isn't in a ready form. But you can download it from our discord, I keep it in the #blue-dream channel which is the name of the overall project. If you do decide to go that route I'd be happy to tell you how it works. It's almost ready but still needs cleaning up. It works obviously!
     
    Last edited by a moderator: Jan 20, 2021
  15. Xiao Hayes

    Xiao Hayes

    Back on track Member
    804
    76
    28
    Bilbao, Spain
    Upgrading my own life to pro edition
    My prayers have finally been heard. Not too long ago I stated that I bought Sega consoles basically to play Sonic and Ecco, so having an Ecco site on the level of Sonic Retro that is active and growing sounds the greatest. Well, I'm sorry I can't contribute, but I'd like to see if this goes wild and Ecco gets fangames and stuff like that. Just for you to know, while I'm a frustrated fanwork developer at least for now, I wanted to create an engine for Sonic fangames that also included Ecco gameplay: imagine Ecco exploring a custom level of Labyrinth zone especially suited for his gameplay, even more now that we know proto labyrinth intended to have air chambers like some Ecco levels. I also have in mind how I would have made Ecco III.

    One question: have you talked with Ed Annunziata? I think he probably will like what you're doing here and help a bit if he can and hasn't yet. Extending the site's scope to his Ecco-but-not projects would also be amazing and give the site relevance with present projects, specially if he eventually manages to get the IP from Sega.
     
  16. Arkonviox

    Arkonviox

    Member
    8
    9
    3
    The goal of this project is to build a brand new ecco engine from scratch, I have reverse engineered most of the graphical formats, and object formats. Most of what I need is mapped and if it isn't mapped I can easily find it.

    I been slowly studying Ecco's movement algorithm as I get ready to implement it into the engine. I wrote this article for a few folks in our discord and hope to elaborate on the topic later when I get more time to study it. I will eventually develop a tutorial and a live demo so people can see it in action, that will eventually be in the works later this year. I have experience in game engine design, I built a simple game engine that implement the basics needed for a game.

    https://eccothedolphin.online/ecco-motion-physics/



    I built that in 2014, it was built completely from scratch only using the XNA library for loading and displaying graphics.

    I have older demos that had sound effects but that was probably my best engine at the time.
    This new engine will put that one to shame as it will use alot of ideas I learned from my mega drive and ecco research. I also made a very simple tech demo on the NES back in 2012.

    As for Ed, I've met him in person at a meet and greet over a year ago. He knows about our project and has been interested in our progress.
    [​IMG]

    I even have a few items signed by him! Including my first Ecco game (the game gear version).
    BTW If you look closely at the following photo, in that frame is a prototype of the original Ecco for game gear!

    IMG_20200414_190135957-small.jpg


    And for those wondering, yes that prototype is dumped too!
    It's one of the earliest Ecco game gear prototypes known to exist!

    https://hiddenpalace.org/Ecco_the_Dolphin_(Jun_14,_1993_prototype)
     
    Last edited: Jan 20, 2021
  17. JaxTH

    JaxTH

    Pudding Deity Oldbie
    9,049
    120
    43
    Los Angeles
    Jack shit.
    For the purposed of Sega Retro we do individual pages. However, it doesn't really matter, we have programs made for us and one of those splits images. Those are here, by the way: https://retrocdn.net/Category:Shared_program_files
     
  18. ICEknight

    ICEknight

    Researcher Researcher
    Just saw this on the media page:
    You guys seem to have forgotten about plain actual preservation here, which would be scanning and posting the raw scans of it, and maybe also stitching together the multiple scans which have to be done for the bigger stuff.
    No cleanups, color correction or any kind of alterations should be done for preservation, you calibrate the scanner and apply the resulting color profile to get the real colors, instead of creating an estimate which may look flashier.

    Raw scans are indeed an important thing to keep safe and share, hence why people involved in game preservation are all uploading the raw scans to archive.org.

    There's a go-to guide for video game scan preservation here, if you guys are interested: https://www.preservegames.org/2018/08/video-game-preservation-scanning-guides.html
     
    Last edited: Feb 15, 2021