Sunfish, Core tools [Compile & Decompile]

Post here basically any programming material that will help other programmers out.
Post Reply
User avatar
JacksonCougar
Huurcat
Posts: 2460
Joined: Thu Dec 06, 2007 11:30 pm
Location: Somewhere in Canada

Sunfish, Core tools [Compile & Decompile]

Post by JacksonCougar »

This project is coded to support Halo 2 Xbox only!
I don't think that will make too many of you weep :wink:

The purpose of the project is to simply decompile a retail multiplayer map, then compile a new map from those tag resources.
The goal was to do this as quickly as possible... I have not reached that point of course, but I am happy enough with the results thus far.

Downloads:
The zip contains the sunfish.dll where the magic happens, as well as a test program to let you play with it right out of the box...
http://www.remnantmods.com/files/jackso ... 3-2011.zip New: version 1.3.0.0
Change Log:
  • Various updates to Tag class
  • Various other changes

http://www.remnantmods.com/files/jackso ... 2-2011.zip version 1.0.0.0

Known issues:
Don't even think about maps with more than one sbsp... I completely lazed out on that. (I'll add support in another version)
Unicode only rebuilds with English strings, other tables are removed
StringIDs are arranged in a new order... ( I don't think this will be an issue: it makes rebuilding much faster)

Suspected issues:
I probably have bugs in my structures, so if you noticed something wrong while looking at the rebuilt map in Entity lemme know...

Suggested usage:
I would really suggest you create a separate partition to decompile onto. You don't need to have it on another disk, but just having a small separate partition seems to speed up decompile a lot. Probably because there's less entries in the filesystem...
User avatar
Ogrish
Posts: 1512
Joined: Wed Dec 12, 2007 2:56 am

Re: Sunfish, Core tools [Compile & Decompile]

Post by Ogrish »

so i decompiled an onyx map, i was expecting problems.

It decopiled to the selected folder, but when i look in the folders, it seem as it created folders for deffernt items but didnt place those items in there folders,
and most folders were empty.

I decided to recompile to a new map on the desktop, it started working, the map icon was created instantly, but after that nothing seem to happen.
I waited, then went for some food, when i got back it seemed to be done, i guess, but it never recompiled.

So just now i decompiled a clean zanzibar, it took 101706 milliseconds. :D

I decided to recompile to a new map called recompiled, zanzabar.scnr,h2tag is not in its file, its in scenarios/multi/zanzibar, so i choose that folder.
Your app doesnt let you select the zanzabar.scnr.h2tag, so i select the folder that host it but nothing happens while i hit compile.

After looking more i see that most files are not in there folders, this has happened with H2core too so maybe its a framework problem or something on my end.
Ill put everthing where it goes and try again later.
User avatar
JacksonCougar
Huurcat
Posts: 2460
Joined: Thu Dec 06, 2007 11:30 pm
Location: Somewhere in Canada

Re: Sunfish, Core tools [Compile & Decompile]

Post by JacksonCougar »

It's also a bit of a 'my-bad' thing... it's creating 2x as many folders as it should... the one's with the extensions are there because of a typo in my code. I didn't add any way to see if it's working or not either. It either works and shows "Done", or not. If it returns to the enabled form, without showing a message box that means that it error-ed out during rebuilding. I'll release a slightly less confusing version once I finish with what I'm working on.
User avatar
Ogrish
Posts: 1512
Joined: Wed Dec 12, 2007 2:56 am

Re: Sunfish, Core tools [Compile & Decompile]

Post by Ogrish »

No prob im just trying to give feed back.
User avatar
Gary
Posts: 1946
Joined: Thu Feb 14, 2008 10:17 pm
Location: USA, FL
Contact:

Re: Sunfish, Core tools [Compile & Decompile]

Post by Gary »

In-case you wanted feedback on decompiling:

Worked quickly, didn't lock up and encountered no problems.
Program version: 1.0.0.0 - 22-02-2011

Map: Stock Zanzibar.map
Time according to the program: 12460 Milliseconds(12.46 Seconds)

Total Files In Decompiled directory:
5024 File(s)
18899 Dir(s)
57,488,398 bytes

Drive:
Separate Drive for decompiled content.
Drive Speed:
Write:~994MB/s
Read: ~995MB/s

CPU: AMD Athlon 64 x2 Dual Core 5200+ clocked at 2.7GHz
RAM: 3GB
OS: Windows 7
User avatar
JacksonCougar
Huurcat
Posts: 2460
Joined: Thu Dec 06, 2007 11:30 pm
Location: Somewhere in Canada

Re: Sunfish, Core tools [Compile & Decompile]

Post by JacksonCougar »

Awesome, thanks for giving it a run guys :]. I don't have enough time to do as much testing as I would like, so anything you catch, I'll try to fix of course. Will be releasing the first tool to work with this pretty soon here I hope :laff:
User avatar
Gary
Posts: 1946
Joined: Thu Feb 14, 2008 10:17 pm
Location: USA, FL
Contact:

Re: Sunfish, Core tools [Compile & Decompile]

Post by Gary »

No problem man, any time.

I should try re-compiling now.
User avatar
JacksonCougar
Huurcat
Posts: 2460
Joined: Thu Dec 06, 2007 11:30 pm
Location: Somewhere in Canada

Re: Sunfish, Core tools [Compile & Decompile]

Post by JacksonCougar »

Update: if you want to re-test it should tell you what's wrong now (exceptions)
User avatar
Gary
Posts: 1946
Joined: Thu Feb 14, 2008 10:17 pm
Location: USA, FL
Contact:

Re: Sunfish, Core tools [Compile & Decompile]

Post by Gary »

All the speeds and settings are the same as they were last test, though this time it about 15 seconds to decompile.

For some reason, your app is looking in the wrong directory.

Code: Select all

************** Exception Text **************
System.IO.DirectoryNotFoundException: Could not find a part of the path 'E:\Users\root\Documents\tags'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Directory.SetCurrentDirectory(String path)
   at Sunfish.Compiler.Initialize()
   at Sunfish.Compiler..ctor(Stream output)
   at TestProject.Main.Compile()
   at TestProject.Main.cmdCompile_Click(Object sender, EventArgs e)
   at System.Windows.Forms.Control.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ButtonBase.WndProc(Message& m)
   at System.Windows.Forms.Button.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4952 (win7RTMGDR.050727-4900)
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll
----------------------------------------
TestProject
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Users/Gary/Desktop/sunfish%20-%20v1.2%20-%20with%20example%20program%20-%2002-03-2011/TestProject.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4927 (NetFXspW7.050727-4900)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4927 (NetFXspW7.050727-4900)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4927 (NetFXspW7.050727-4900)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
Sunfish
    Assembly Version: 1.2.0.0
    Win32 Version: 1.2.0.0
    CodeBase: file:///C:/Users/Gary/Desktop/sunfish%20-%20v1.2%20-%20with%20example%20program%20-%2002-03-2011/Sunfish.DLL
----------------------------------------
User avatar
JacksonCougar
Huurcat
Posts: 2460
Joined: Thu Dec 06, 2007 11:30 pm
Location: Somewhere in Canada

Re: Sunfish, Core tools [Compile & Decompile]

Post by JacksonCougar »

Oh hai, that's my directory :l0l:

I'll fix that...
User avatar
kornman00
Posts: 104
Joined: Wed Jan 20, 2010 7:48 pm

Re: Sunfish, Core tools [Compile & Decompile]

Post by kornman00 »

You may wish to look at my BlamLib libraries. The codebase is setup and works with all current games. However, when it comes to tag extraction, it extracts tags into the official editor's (currently only Halo1 and Halo2) tag file formats. I've already added tag reconstruction support for such things like render_models, bitmaps, unicode_string_lists, etc. The library was made with the mentality of taking game data (ie, cache files and their tag data) and recreating source tags that you can then use to create maps with the official tools, instead of working directly on cache files (doing the latter is HIGHLY counter-intuitive for anything after Halo1 based engines due to data and format optimizations).

Side note on your string_id reordering: the engine assigns fixed IDs to a range of string_ids (as those IDs are used in the engine's code), so you can't change the order of those. However, any string_ids which are specific to the cache can be in any order (as long as that order is consistent with all string_id fields in all the tag definitions). To give an example, here is the definition list for the string_ids which the Halo2_Xbox engine defines. These strings and their values (IDs) must always be in the order given.
User avatar
DoorM4n
Posts: 2154
Joined: Sun Dec 09, 2007 3:01 am
Location: Houston

Re: Sunfish, Core tools [Compile & Decompile]

Post by DoorM4n »

Dang man! This is really awesome. Keep up the work and know that this tool would do a lot of wonderful things for halo modders.
Image
Remnant! We were the last stand.
Post Reply