don't click here

Utility ClownMapEd - The Open Source SonMapEd Clone

Discussion in 'Engineering & Reverse Engineering' started by Clownacy, Apr 19, 2023.

  1. Clownacy

    Clownacy

    Tech Member
    1,061
    608
    93
    SonMapEd is pretty old, having last been updated in 2007, but for the last decade it has served as my go-to sprite editor and converter. Unfortunately, there are a few places where it shows its age, such as its inefficient data compression, blurriness on high-DPI displays, and shoddy DPLC generation. Due to SonMapEd being closed-source software, it is not possible to address these issues without complex low-level reverse engineering of the EXE. Additionally, being nothing more than a closed-source EXE, SonMapEd lacks native support for other operating systems (such as Linux) and CPU architectures (such as ARM).

    I hope to bring an end to these problems and more with the release of my open source, portable SonMapEd clone - ClownMapEd!

    Features:
    • Written in modern C++!
    • Support for Windows, Linux, and presumably many other desktop operating systems, owing to the portability of the Qt framework!
    • Support for high-DPI displays, eliminating the blurriness that plagues SonMapEd!
    • Vastly-improved DPLC generation, producing data that is even more efficient than that in the original games, and far more efficient than that of SonMapEd!
    • Improved data compression, by leveraging Flamewing's 'mdcomp' compression library!
    • Support for the Kosinski+ and Comper compression formats!
    • Improved colour-matching for sprite importing, leveraging the Delta-E 2000 colour difference algorithm!
    • Additional keyboard shortcuts that benefit users of alternate keyboard layouts!
    • An accurate clone of SonMapEd, meaning that anyone who is familiar with it will be right at home with ClownMapEd!
    • Not based on the bloated Electron framework, unlike a certain other open source sprite editor! ;)

    Screenshot:
    [​IMG]

    Source Code: https://github.com/Clownacy/clownmaped
    Windows Executable & Linux AppImage: https://github.com/Clownacy/clownmaped/releases
     
    Last edited: Apr 12, 2024
    • Like Like x 18
    • Useful Useful x 2
    • List
  2. Chimes

    Chimes

    The One SSG-EG Maniac Member
    640
    489
    63
    Oh my gooood please dont say sike dont let this be a dream
    This is truly a godsend tool
     
  3. MarkeyJester

    MarkeyJester

    Original, No substitute Resident Jester
    2,209
    437
    63
    Japan
    This is so surreal and quite the bloody match of accuracy, I'm impressed!

    I have a few suggestions, I hope you're taking some...
    • Support for "Organise All Tiles by Sprite Order" like SonMapEd (the menu didn't mention it, but there is a short-cut key too; "O").
    • Zoom support via +/- and CTRL +/-, for top and bottom sections.
    • The ability to open uncompressed art files with ClownMapEd via "open with"/association, and have it automatically load in from the beginning for quick tile viewing. (I'm aware there is no way to tell if the file being opened is an uncompressed art file, it'd be entirely at user discretion, this is something SonMapEd annoyingly didn't have and I wish it did, would be bloody useful, even if you leave the mappings/plc/palette alone which were previously there).
    • Rulers (or at least a crosshair which stands out) to help identify the centre of the object, this is one of SonMapEd's biggest flaws, you have to fiddle to locate the relative position.
     
    • Agree Agree x 5
    • Like Like x 1
    • List
  4. Clownacy

    Clownacy

    Tech Member
    1,061
    608
    93
    I did try to add the 'Organise All Tiles by Sprite Order' button, but there was an edge-case that I couldn't find a decent way to handle. Not even SonMapEd handles it properly.

    Say you have two sprites, the first one having a single 1x1 sprite piece that starts and ends with tile 1, and the second having a single 1x2 sprite piece that starts with tile 0 and ends with tile 1. On one hand, it would be impossible to organise the tiles by sprite order because the second sprite's first tile must come before the first sprite's only tile, but, on the other hand, it would be perfectly acceptable to have them in an incorrect order in this specific case anyway. SonMapEd does not do even this, however, and instead creates a duplicate of tile 1 so that it can go before the second sprite's tile 0 and 1.

    The trouble is coming up with a good algorithm that properly addresses this problem.
     
    Last edited: Apr 20, 2023
  5. MarkeyJester

    MarkeyJester

    Original, No substitute Resident Jester
    2,209
    437
    63
    Japan
    Not really asking for actual sprite ordering, more if you'd support showing the tiles vertically in the tile viewer, row count dictated by window height. I used SonMapEd's misnomer "Organise All Tiles by Sprite Order" simply to communicate the function it happens to perform, apologies for the confusion.

    Let me rephrase; "Support for showing tiles vertically in the tile viewer, short-cut key "O" to switch". It sounds silly, but it's very useful for... certain things...
     
  6. Clownacy

    Clownacy

    Tech Member
    1,061
    608
    93
    Oh, that - that's the 'Orient Tiles Vertically' option, which is found under 'Settings/Tile Rendering'. I'll be sure to add that (as well as other missing SonMapEd features) in an update.
     
  7. MarkeyJester

    MarkeyJester

    Original, No substitute Resident Jester
    2,209
    437
    63
    Japan
    Oh geez, hahahaha, forgive me. For some reason all these years I had that option name associated with that function in my head, no wonder it didn't label "O" as the short-cut in the menu, I guess it's not a misnomer after all. Forgive my ignorance d=

    But yeah, if you could support that, that would be absolutely swell!
     
  8. Selbi

    Selbi

    The Euphonic Mess Member
    1,501
    52
    28
    Northern Germany
    Sonic ERaZor
    If there's one thing I'd love to see added, it's the ability to change hotkeys. The original SonMapEd was clearly not designed with foreign keyboard layouts in mind, as apparent by the rampant usage of [ and ] to navigate through the tool:

    upload_2023-4-22_17-47-55.png

    The problem is that it's physically impossible to press Ctrl+] on a German keyboard, as we poor souls already need to hold down a special key called Alt Gr to even access [ and ] keys.

    I still remember manually clicking through this menu over and over to swap sprites all the way to the end of the list. It was a nightmare.
     
    • Agree Agree x 3
    • Informative Informative x 1
    • List
  9. President Zippy

    President Zippy

    Zombies rule Belgium! Member
    Good stuff, Maynard G! I appreciate the focus on portability, it makes me feel less left-out as a FreeBSD and OpenBSD user.

    Man, I remember when I discovered SonED2 10-12 years ago (running on Windows 7) and used it to make a simple layout hack of Labyrinth Zone. My hack sucked eggs and suffered from slowdown in Kega Fusion, but at least I could play something. ACM should have given an award to the guy who reverse engineered the tile compression algorithm.
     
  10. OrionNavattan

    OrionNavattan

    Tech Member
    171
    166
    43
    Oregon
    Quickly ran into a a bit of a compatibility issue: the program seems to only allows loading tile data from files with a .bin extension. While this is accurate to SonMapEd, it prevents the program from being used with Hivebrain's and my own disassemblies of Sonic and Sonic 2, as we use .nem, .kos, and .unc for our tile files.

    On a side note, are there any plans to add support for loading or exporting macro-based mappings, specifically spritepiece and the custom format in HB2022?
     
  11. Clownacy

    Clownacy

    Tech Member
    1,061
    608
    93
    Arg, I knew the '.bin' thing would catch up to me eventually. Even the FW-KENSC shell extension that me and MainMemory made uses '.nem' and '.kos' and the like. I'll release a hotfix that addresses that soon.

    I would like to add support for macro-based mappings. It should be easy enough to add, assuming it doesn't require label names to be preserved (neither SonMapEd nor ClownMapEd currently preserve the label names of assembly-based mappings).
     
    Last edited: Apr 27, 2023
  12. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,744
    338
    63
    SonLVL
    There are certain specific cases where labels are important, such as Sonic's running frames in S2.
     
  13. OrionNavattan

    OrionNavattan

    Tech Member
    171
    166
    43
    Oregon
    Exactly what I was going to mention: the DPLCs for Sonic's running frames in Sonic 2 (which are referenced directly by the Sega screen). HB2022 takes it to a whole different level: it explicitly depends on the labels in the mappings files, as the ID constants generated by its index table system are used to build the animation scripts and handle any instances where objects need to set or check for a specific frame (as will mine once it's finished):

    Code (ASM):
    1.  
    2. Walk:       dc.b $FF
    3.        dc.b id_frame_walk13
    4.        dc.b id_frame_walk14
    5.        dc.b id_frame_walk15
    6.        dc.b id_frame_walk16
    7.        dc.b id_frame_walk11
    8.        dc.b id_frame_walk12
    9.        dc.b afEnd
    10.        even
    11.  
     
  14. VAdaPEGA

    VAdaPEGA

    Freelance Digital Artist Member
    80
    23
    8
    Portugal
    Probably something minimalistic
    This is a problem on the portuguese and a good majority of other EU keyboard layouts where [ and ] are stuffed in the numeric keys and therefore don't work as shortcuts, my workaround is to switch between keyboard layouts on the spot which tends to make me write ' as - in chats which is rather annoying.

    Oh and please, for the sake of not having to use the dumb save state feature, add an undo feature.
     
  15. Clownacy

    Clownacy

    Tech Member
    1,061
    608
    93
    Here is v1.0.1. It is a small update that adds two things:
    • More file extensions, such as '.nem' and '.kos'.
    • More keyboard shortcuts, which provide alternatives to the ';', ''', '[', and ']' keys. These new shortcuts use the '6', '7', '8', and '9' keys, respectively.
     
  16. Clownacy

    Clownacy

    Tech Member
    1,061
    608
    93
    Here is v1.1. It fixes a couple of bugs and adds support for MainMemory's MapMacros, which the Git disassemblies of Sonic 1 and 2 now use.
     
  17. Selbi

    Selbi

    The Euphonic Mess Member
    1,501
    52
    28
    Northern Germany
    Sonic ERaZor
    I've recently messaged you on Discord about this without a response, so I'll repeat it here again since I think it's a rather inconvenient shortcoming compared to the old version of SonMapEd.

    So basically, this tool is subpar for the old 2005 disassembly. Its reliance on macros to do the job leads to various problems. Naturally, I don't have the MapMacros installed, so I unchecked that, but that didn't help much either:

    [​IMG]

    ClownMapEd uses .offsets and .frameN for its purposes. While clean in code, it causes label collisions elsewhere. SonMapEd circumvented this problem with a rather ugly but effective method, which is generating a random string when saving maps:

    upload_2023-11-25_15-49-39.png

    "A2xNg" in this case. Really ugly, but it gets the job done.

    I'd love to see this as a legacy feature added, disabled by default of course since I'm well aware it's not the recommended approach anymore, but it gives us poor people who started working on their projects back when the 2005 disassembly was all we got some relief. Right now I have to manually adjust the mapping files each time I save them and it's rather time consuming.

    EDIT: Oh yeah one more thing, while importing sprites it uses the background color of whatever palette is currently active, while the interface still uses the background color of line 1. Took me a while to figure out and would be nice to see adressed.
     
  18. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,744
    338
    63
    SonLVL
    Sounds like the issue is rather that the program is assuming you're using the AS assembler, not ASM68K. Swapping all the
    Code (Text):
    1. .
    with
    Code (Text):
    1. @
    would fix it for you.
     
    • Informative Informative x 1
    • List
  19. Devon

    Devon

    A̸ ̴S̴ ̵C̵ ̷E̶ ̸N̸ ̴D̶ ̵E̶ ̸D̶ Tech Member
    1,265
    1,433
    93
    your mom
    Alternatively, you can set an assembler option for ASM68K to change the local label symbol to a "." by adding "/o l." in the command line, or adding "opt l." in the code.
     
    • Informative Informative x 1
    • List
  20. Selbi

    Selbi

    The Euphonic Mess Member
    1,501
    52
    28
    Northern Germany
    Sonic ERaZor
    Regardless of which approach should be taken, this would've been great to know beforehand.