don't click here

Help request: C-Programmer to add new function to WLA DX

Discussion in 'Engineering & Reverse Engineering' started by Kroc, Dec 6, 2013.

  1. Kroc

    Kroc

    Code is Art Tech Member
    38
    1
    8
    MaSS1VE: The Master System Sonic 1 Visual Editor
    Hello.
    I am working on a complete disassembly of Sonic 1 Master System for the purposes of a full editing suite for the game.

    I am working toward porting the ROM from 256K to 512K. This requires properly labelling and structuring the disassembly so that the code and data is portable and can be moved around the different banks.

    I am using WLA DX and whilst it is an excellent tool for the job I am coming across instances of code and data that are not simple enough to make portable.

    I would like to ask for the help of a C-programmer to add some new functions to WLA DX in order to solve some portability problems in the code.

    Here's my first proposal for a new function:

    ".TABLE": Portable indexes

    Here's a real-world problem I want to solve: making index numbers used in data, portable. In Sonic 1 SMS each level has a list of which objects (enemies, powerups &c.) appear where in the level. In the ROM a pointer table provides which object index number runs which code. This style of index-based functionality is very common, I imagine in most other games too.

    How can we add to / remove from / rearrange the object code and function pointer list in the game without making all the level data invalid; pointing to the wrong objects? The current situation is that nobody bothers and just works around the ugly picture. The lists are included as binary and the pointer tables are only appended to. This is not flexible enough for me, especially if I want to head toward providing an editing suite with total-conversion capabilities.

    My proposal is a new declaration for WLA DX: ".TABLE".

    First you define the 'columns' of your table using a standard .STRUCT definition.

    Code (Text):
    1. .STRUCT pointerList
    2.     address: dw
    3. .ENDST
    Here's where the magic happens. Each row has it's own label, this label does not point to the output address of that data, but instead provides the index number of the row. I.e. 'badnick_motobug` = 2

    Code (Text):
    1. .TABLE objectPointers INSTANCEOF pointerList
    2.     .ROW badnick_crabmeat DATA $1234
    3.     .ROW badnick_chopper  DATA $5678
    4.     .ROW badnick_motobug  DATA $9ABC
    5.     [...]
    6. .ENDT
    Now in your data statements you can refer to these labels rather than hard values

    Code (Text):
    1. ;list of objects in the level: (object ID, X, Y)
    2. .db badnick_chopper, $20, $20
    3. .db monitor_ring, $43, $12
    4. [...]
    .TABLE is also excellent for data that combines different data types (bytes, words, strings &c.). Currently in WLA DX this quite painful, e.g.

    Code (Text):
    1. .db $FF
    2. .dw $1234
    3. .db $80
    4. [...repeat 100 times...]
    Doing this with .TABLE will be far more elegant and flexible:

    Code (Text):
    1. .STRUCT hypotheticalTableDef
    2.     first: db
    3.     second: dw
    4.     third: db
    5. .ENDST
    6.  
    7. .TABLE hypotheticalTable INSTANCEOF hypotheticalTableDef
    8. .ROW DATA $FF, $1234, $80
    9. .ROW DATA $11, $5678, $A0
    10. [...]
    11. .ENDT
    .TABLE would also define `_sizeof_[table name]` which will give how many rows are in the table -- perfect for initialising your loop in a flexible, portable way!

    [hr]

    I also have a proposal for a block form of .DSTRUCT that will help with self-documentation, but I will post details of this soon.

    Is there any one out there that would be willing to volunteer their efforts to help?
     
  2. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,742
    338
    63
    SonLVL
    I don't know about WLA DX, but in AS your problem could be solved with macros.
     
  3. Kroc

    Kroc

    Code is Art Tech Member
    38
    1
    8
    MaSS1VE: The Master System Sonic 1 Visual Editor
    I must admit that I am new to WLA DX, since I am likewise new to Z80 which I have taken to learning for the purpose of this project. I've read over the WLA DX documentation several times and I am not aware of a way that I could replicate the desired functionality _elegantly_. If someone can show me otherwise, I'd be glad as it would mean less work for everybody involved.
     
  4. Glitch

    Glitch

    Tech Member
    175
    12
    18
    Something like this should work:

    Code (Text):
    1.  
    2.     .ORG $200
    3.  
    4.     .MEMORYMAP
    5.         SLOTSIZE    $4000
    6.         SLOT        0 $0000
    7.         SLOT        1 $4000
    8.         SLOT        2 $8000
    9.         DEFAULTSLOT 0
    10.     .ENDME
    11.  
    12.     .ROMBANKMAP
    13.         BANKSTOTAL  16
    14.         BANKSIZE    $4000
    15.         BANKS       16
    16.     .ENDRO
    17.  
    18.     .MACRO def_object ARGS obj_name, logic_ptr
    19.     _def_object_\@:
    20.         .dw \2
    21.         .def \1 (_def_object_\@ - OBJ_TABLE_START)/2
    22.         .export \1
    23.     .ENDM
    24.  
    25.     OBJ_TABLE_START:
    26.         def_object  object1, object1_handler
    27.         def_object  object2, object2_handler
    28.         def_object  object3, object3_handler
    29.  
    30.  
    31.     object1_handler:
    32.         ld a, object1
    33.         ret
    34.  
    35.     object2_handler:
    36.         ld a, object2
    37.         ret
    38.  
    39.     object3_handler:
    40.         ld a, object3
    41.         ret
    42.  
    43.  
    Which generates the following symbols:

    Code (Text):
    1.  
    2.     0000:0000 object1
    3.     0000:0001 object2
    4.     0000:0002 object3
    5.     0000:0200 OBJ_TABLE_START
    6.     0000:0200 _def_object_0
    7.     0000:0202 _def_object_1
    8.     0000:0204 _def_object_2
    9.     0000:0206 object1_handler
    10.     0000:0209 object2_handler
    11.     0000:020c object3_handler
    12.  
    13.  
     
  5. Kroc

    Kroc

    Code is Art Tech Member
    38
    1
    8
    MaSS1VE: The Master System Sonic 1 Visual Editor
    That'll work, but it won't suit my needs:

    * It requires a macro for each instance of a table rather than for each definition

    * It doesn't define _sizeof_tablename automatically, that would have to be maintained manually which I'm looking to avoid

    * It's less readable, less intuitive and would make the source more difficult to learn. It's really important to me to maximise accessibility. I would want anybody like myself who was interested and was learning Z80 to have an easy to understand and descriptive piece of code
     
  6. Kroc

    Kroc

    Code is Art Tech Member
    38
    1
    8
    MaSS1VE: The Master System Sonic 1 Visual Editor
    I'm trying to solve the sizeof problem with macros, but I can't see a way in WLADX to reference the name of one variable using another. I.e.

    Code (Text):
    1. .MACRO TABLE ARGS tableName
    2.     ;define the current position as the table name
    3. __TABLE\@__:
    4.     .DEF \1 __TABLE\@__
    5.     ;then define a reference used for counting the row index
    6.     .REDEF __TABLE_size__ 0
    7.     ;define the size of the table
    8.      ;we need to increase this with each 'ROW` macro
    9.     .DEF _sizeof_\1 0
    10. .ENDM
    11.  
    12. .MACRO ROW ARGS rowIndexLabel
    13. __ROW\@__:
    14.     .IFDEFM \1
    15.         .DEF \1 __TABLE_size__
    16.     .ENDIF
    17.     ;increase the index number
    18.     .REDEF __TABLE_size__ (__TABLE_size__+1)
    19.     ;how do we increase '_sizeof_[tableName]` without
    20.      ;having the table name as an argument?
    21. .ENDM
    It could be possible by having an 'ENDT` macro using the table name, but I'd rather avoid having to give the tablename twice, the reason why I'd prefer changes to WLA DX rather than using macros alone.
     
  7. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,742
    338
    63
    SonLVL
    Rather than trying to find someone willing and able to rewrite parts of WLA DX, you could try using AS, which already allows you to construct symbol names using other symbols.
     
  8. Kroc

    Kroc

    Code is Art Tech Member
    38
    1
    8
    MaSS1VE: The Master System Sonic 1 Visual Editor
    Stupid question, but what's AS?
     
  9. flamewing

    flamewing

    Emerald Hunter Tech Member
    1,161
    65
    28
    France
    Sonic Classic Heroes; Sonic 2 Special Stage Editor; Sonic 3&K Heroes (on hold)
    AS is the macro assembler (here). It is a multiprocessor assembler which is used in the S2 and S3&K disassemblies because it can assemble their z80 sound drivers alongside their m68k main code.
     
  10. Kroc

    Kroc

    Code is Art Tech Member
    38
    1
    8
    MaSS1VE: The Master System Sonic 1 Visual Editor
    Thank you for pointing me to that @flamewing. I've discovered many more bugs and limitations of WLA DX that I don't think would be solvable any time soon and whilst AS is nice I feel it suffers from the same design problems with WLA DX that I'm looking to overcome.

    To that end I'm formulating a spec for a new ASM script language, here's a very rough draft to show what I'm getting at:
    https://gist.github.com/Kroc/7940880
     
  11. MainMemory

    MainMemory

    Kate the Wolf Tech Member
    4,742
    338
    63
    SonLVL
    Writing an assembler is hard work that not everyone can do, and those who can only will if they feel that the current assemblers are lacking features or have bugs that they cannot work around. Then if you add to that most people are only interested in the Mega Drive games, the number of people willing to do this becomes even smaller.

    I will say though that FraGag is currently working on a new assembler architecture, and given that the MD has a Z80 processor, he'll probably add support for it eventually. Current estimations put the first release sometime within the next century. :v:/>

    It is probably possible to get all of the functionality you want with some sort of macros, but I don't know anything about SMS or Z80 so I won't be much help unfortunately.

    Edit: If you want a type of table that automatically assigns IDs, you could have a macro at the start of the table that sets a variable to 0, then each entry is a macro either with a label that becomes the ID constant on the line or a string parameter that names the ID constant, and all the data for the structure. The macro sets the ID constant to the current value of the counter variable, then increases the variable by 1.
     
  12. Dr. Kylstein

    Dr. Kylstein

    Member
    86
    0
    6
    Perhaps a pre-processor is in order? I remember reading about somebody piping their assembly through gpp (GNU C PreProcessor) to handle a problem like this. Or you might have an easier time finding a Python [or Perl, or whatever] programmer to make a custom pre-processor than a C programmer willing to create/hack an assembler.
     
  13. Kroc

    Kroc

    Code is Art Tech Member
    38
    1
    8
    MaSS1VE: The Master System Sonic 1 Visual Editor
    There are limitations in WLA DX that would prevent that from solving some problems -- I'll just copy in the reply I made on the SMSPower thread where this is also being discussed.

    [hr]

    I'll give you an example where WLA DX is slipping up and where a preprocessor really won't help --

    I need to calculate offsets from the start of a bank to the data, but in a portable way since I will be providing 256K and 512K ROM layouts.

    You can't calculate using a bank number, nor can you pass it as a macro parameter.

    Code (Text):
    1. _label:
    2. .PRINTV HEX :_label ;this does not work
    A preprocessor cannot solve this as it cannot tell what bank a symbol may lie in.

    Here's another example of something that really badly needs fixing:

    Bank boundaries are a pain to manage. For example, the block mappings in Sonic 1 extend from $10'000 to $15'580, across the bank boundary at $14'000. If I (or anybody) wants to modify that data, they have to manually move the .BANK statement precisely byte-for-byte.

    I propose the ability to allow data to not overflow bank boundaries when specified in this manner:

    Code (Text):
    1. .BANK 3, 4 ;or, also like
    2. .BANK 0-2
    This way I can have data that crosses the bank boundary without having to manage the exact byte split.

    [hr]

    Yes, and also no-no-no :) I've got some macros that largely implement the table syntax, but I'm running up against WLA DX bugs and outright holes in functionality that cannot be macro'd or preprocessored around.

    I appreciate all the words and feedback. I'm rolling everything around in my head and working out the best strategy.