H2PC Guerilla tag layouts

This is dedicated to all the mods the community wants to share!
Patches only people. No Map files.
Post Reply
User avatar
Grimdoomer
Admin
Posts: 1835
Joined: Sun Dec 09, 2007 9:09 pm

H2PC Guerilla tag layouts

Post by Grimdoomer »

While working on my map compiler I found I needed a better way to look at the tag layouts the PC counterpart's Guerilla used to create and edit tags. I used some code from Kornman's BlamLib and OpenSauce libraries and made a quick tool to extract the tag layouts and put them into a format that somewhat resembles the c-style format Bungie originally used for the pc tool set.

What does this mean? Not a whole lot, really. The layouts I extracted include some information not present in other dumps I have made, such as sizes and alignment intervals for tag_blocks, the name of the structures certain fields like tag_block indices and tag_block flags point to, addresses of misc functions, etc. These aren't terribly useful to the average person, and if interpreted wrong could lead to breaking shit.

A few things to note about these tag layouts. Firstly they are from the pc tool set, they are intended for editing tag files before they get compiled into a working map. They layouts are saved as C header files, although they are not real header files, these are not the names of the header files this data would have been stored in. Some tags and some tag_blocks have multiple field sets associated with them, meaning they have more than one layout. I chose to use what is defined as the "latest" layout to create these tag layouts with. The output can be found in the file "log.txt".

Code: Select all

- The options for enums and bitmasks are separated by a ":" character.

- _field_terminator is just used to signify the end of a list of fields, it has no data associated with it. 

- I don't believe the field type _field_useless_pad is actually padding, I believe it is a legacy field left in the layouts that has a size of 0, although I'm not positive of this.

- FIELD_EXPLAINATION is just used for information in the GUI interface, it is not real data. If you see "<lb>" in an explanation it is my special character for a line break.

- _field_custom is used for functions, the only data the meta information has for it is an address of a function that gets called to do something relating to the function. I'm not sure of the actual size of this field, or if it is constant for all instances of it.

- _field_array_start is an array of elements, the elements being whats between the _field_array_start field and the _field_array_end field. The number associated with the _field_array_start field is the number of entries in the array.

- _field_char_block_index1/_field_short_block_index1/_field_long_block_index1 fields are indices into tag_blocks, the data associated with them is the name of the tag_block structure they reference.

- _field_char_block_index2/_field_short_block_index2/_field_long_block_index2 work similar to the above mentioned tag_block indices, except there is a function address used to get the block index, and a function address used to check the block selected is valid.

- _field_char_integer/_field_short_integer/_field_long_integer if these fields have extra data associated with them it means they are being used as a tag_block index, they work the same way as the previously mentioned block index fields.

- _field_byte_block_flags/_field_word_block_flags/_field_long_block_flags these fields work similar to tag_block indices, except they point to a flag field within the tag_block.
I think that is about it. I have attached the tag layouts as well as the C# source files needed to dump them yourselves. The code is a hack job but a functioning one so don't judge.
Attachments
tag_groups_latest.rar
tag layouts
(271.1 KiB) Downloaded 398 times
GuerillaRipper.rar
source to dump layouts
(5.71 KiB) Downloaded 372 times
Don't snort the magic, we need it for the network.
User avatar
kornman00
Posts: 104
Joined: Wed Jan 20, 2010 7:48 pm

Re: H2PC Guerilla tag layouts

Post by kornman00 »

_field_useless_padding is used to denote debug (ie, tags-builds) only padding (used to add new fields to existing structs). Those padding fields are compiled out of the C struct in release/cache builds. They're required information for tag files and tags in the H2 alpha.

_field_custom is just markup, like _field_explanation. It's used to define where to insert special editor-only controls (eg, Show Bitmaps in the bitmap group)

You pretty much have block_index2 down. Just want to further note that it is used when a direct block element reference can't be gathered just from the index value alone. Eg, as is the case scenario_object_name

I don't recall _integer fields having any values for their definition, let alone a reference to a tag_block_definition

block_flags doesn't reference a flags field, the field itself represents a bitvector of block indices. Eg, Manual BSP Flags specifies which block indices to use the scenario object datum on. It's more efficient memory and processing wise than defining a tag_block representing the actual block_indexes. Of course, it's limited to tag_blocks which have a max element count that is <= LONG_BITS (32)

If you ever touch the code again, I'd suggest keeping track of which definitions (struct/block/tag reference/enum/flags) you've already dumped. Eg, you repeat the scenario_object_datum struct in the scenario.
User avatar
JacksonCougar
Huurcat
Posts: 2460
Joined: Thu Dec 06, 2007 11:30 pm
Location: Somewhere in Canada

Re: H2PC Guerilla tag layouts

Post by JacksonCougar »

these are not the same as xbox eh?
User avatar
JacksonCougar
Huurcat
Posts: 2460
Joined: Thu Dec 06, 2007 11:30 pm
Location: Somewhere in Canada

Re: H2PC Guerilla tag layouts

Post by JacksonCougar »

Grim did they remove SP related things from H2Guerilla? Just looking at the sacenario layout you ripped the enum strings for the maptype (Type) are string.Empty, Multiplayer, string.Empty, string.Empty, string.Empty

Code: Select all

	{_field_enum, "Type", ":Multiplayer:::"},
and the PlacementFlags in the scenario_object_datum_struct_block seem to have the Singleplayer spwning flags names set to 'Unused'

Code: Select all

{_field_long_flags, "Placement Flags", "Not Automatically:Unused:Unused:Unused:Lock Type to Env. Object:Lock Transform to Env. Object:Never Placed:Lock Name to Env. Object:Create at Rest"},
and why is the size of scenario_object_permutation_struct_block set to 24?

Code: Select all

                /****************************************
		* size:	24
		* alignment:	0
		* max_count:	1
		*
		****************************************/
		{_field_struct, "Permutation Data", &scenario_object_permutation_struct_block},
		TAG_BLOCK(scenario_object_permutation_struct_block, 1, sizeof(s_scenario_object_permutation))
		{
			{_field_string_id, "Variant Name"},
			{_field_long_flags, "Active Change Colors", "Primary:Secondary:Tertiary:Quaternary"},
			{_field_rgb_color, "Primary Color"},
			{_field_rgb_color, "Secondary Color"},
			{_field_rgb_color, "Tertiary Color"},
			{_field_rgb_color, "Quaternary Color"},
			{_field_useless_pad, 16},
			{_field_terminator}
		};
Add up the fields and I get: 4 + 4 + 3 + 3 + 3 + 3 = 20 (the useless pad would make this larger so I ignored it)
20 just happens to be what I have to set the size to to make the entire struct 'right' for xbox.

Actually I'm willing to go out on a limb and say they probably just padded the rgb_color field with a byte to bring it up to cache alignment on the pc... which would explain the size.
User avatar
XZodia
Staff
Posts: 2208
Joined: Sun Dec 09, 2007 2:09 pm
Location: UK
Contact:

Re: H2PC Guerilla tag layouts

Post by XZodia »

JacksonCougar wrote:Grim did they remove SP related things from H2Guerilla?
Singleplayer info was removed from the scnr, because it wasnt intend to be used for singleplayer maps, but the other tags are just disabled so should be fine.
JacksonCougar wrote:and why is the size of scenario_object_permutation_struct_block set to 24?
JacksonCougar wrote:Actually I'm willing to go out on a limb and say they probably just padded the rgb_color field with a byte to bring it up to cache alignment on the pc... which would explain the size.
This is probably accurate, unless grim's code just calculated it wrong? Counting the colors as size 4 instead of 3.
Image
JacksonCougar wrote:I find you usually have great ideas.
JacksonCougar wrote:Ah fuck. Why must you always be right? Why.
User avatar
Grimdoomer
Admin
Posts: 1835
Joined: Sun Dec 09, 2007 9:09 pm

Re: H2PC Guerilla tag layouts

Post by Grimdoomer »

kornman00 wrote:_field_useless_padding is used to denote debug (ie, tags-builds) only padding (used to add new fields to existing structs). Those padding fields are compiled out of the C struct in release/cache builds. They're required information for tag files and tags in the H2 alpha.

_field_custom is just markup, like _field_explanation. It's used to define where to insert special editor-only controls (eg, Show Bitmaps in the bitmap group)

block_flags doesn't reference a flags field, the field itself represents a bitvector of block indices. Eg, Manual BSP Flags specifies which block indices to use the scenario object datum on. It's more efficient memory and processing wise than defining a tag_block representing the actual block_indexes. Of course, it's limited to tag_blocks which have a max element count that is <= LONG_BITS (32)
This is good to know. I haven't touched any code that deals with reading or saving tag files in guerilla yet. I've only lightly touched the code to create a new tag mainly to see how it handles creating a datum_index for it so it can be tracked.
kornman00 wrote: I don't recall _integer fields having any values for their definition, let alone a reference to a tag_block_definition
I only encountered one instance where this occurs, in the scenario_structure_bsp definition. Take a look at the following tag_block, specifically the fields "Back Cluster" and "Front Cluster":

Code: Select all

/****************************************
* size:	40
* alignment:	0
* max_count:	MAXIMUM_CLUSTER_PORTALS_PER_STRUCTURE = 512
*
****************************************/
{_field_block, "Cluster Portals*", &structure_bsp_cluster_portal_block},
TAG_BLOCK(structure_bsp_cluster_portal_block, MAXIMUM_CLUSTER_PORTALS_PER_STRUCTURE, sizeof(struct cluster_portal))
{
	{_field_short_integer, "Back Cluster*", &structure_bsp_cluster_block},
	{_field_short_integer, "Front Cluster*", &structure_bsp_cluster_block},
	{_field_long_integer, "Plane Index*"},
	{_field_real_point_3d, "Centroid*"},
	{_field_real, "Bounding Radius*"},
	{_field_long_flags, "Flags*", "AI Cannot Hear through This:One-Way:Door:No-Way:One-Way Reversed:No One Can Hear through This"},

        .......
};
kornman00 wrote: If you ever touch the code again, I'd suggest keeping track of which definitions (struct/block/tag reference/enum/flags) you've already dumped. Eg, you repeat the scenario_object_datum struct in the scenario.
Yeah this was definitely not the best way to do it, but for the sake of anyone else being able to read them the way they are use to reading a plugin it seemed to be the most "robust" solution at the time. I would really like to try and create an actual set of C structs from these definitions at one point.
JacksonCougar wrote:Grim did they remove SP related things from H2Guerilla?
They may have just used a separate version for the scenario definition, and the one I ripped had it removed. I can check if I find some time.
JacksonCougar wrote:and why is the size of scenario_object_permutation_struct_block set to 24?

Add up the fields and I get: 4 + 4 + 3 + 3 + 3 + 3 = 20 (the useless pad would make this larger so I ignored it)
20 just happens to be what I have to set the size to to make the entire struct 'right' for xbox.

Actually I'm willing to go out on a limb and say they probably just padded the rgb_color field with a byte to bring it up to cache alignment on the pc... which would explain the size.
rgb_color is 4 bytes, not 3. If you count it as 4 bytes it matches fine. Also the sizes are not calculated, they are stored as meta data in the layouts themselves. They would have been computed by:

Code: Select all

sizeof(struct structure_definition)
when being compiled. So what you see for the sizes is what the compiler determined as the size, not my tool.
Don't snort the magic, we need it for the network.
User avatar
JacksonCougar
Huurcat
Posts: 2460
Joined: Thu Dec 06, 2007 11:30 pm
Location: Somewhere in Canada

Re: H2PC Guerilla tag layouts

Post by JacksonCougar »

Good to know; xbox version is 3 bytes. Do you know if they changed the fields in any of the structs between xbox and PC?
User avatar
kornman00
Posts: 104
Joined: Wed Jan 20, 2010 7:48 pm

Re: H2PC Guerilla tag layouts

Post by kornman00 »

rgb_color has always been 4 bytes. The alpha byte is unused by code and is treated as padding by the system (has the added benefit of adding alpha in the future). It's more than likely typedef'ed using argb_color in the engine's code.
Grimdoomer wrote:
kornman00 wrote: I don't recall _integer fields having any values for their definition, let alone a reference to a tag_block_definition
I only encountered one instance where this occurs, in the scenario_structure_bsp definition. Take a look at the following tag_block, specifically the fields "Back Cluster" and "Front Cluster":

Code: Select all

/****************************************
* size:	40
* alignment:	0
* max_count:	MAXIMUM_CLUSTER_PORTALS_PER_STRUCTURE = 512
*
****************************************/
{_field_block, "Cluster Portals*", &structure_bsp_cluster_portal_block},
TAG_BLOCK(structure_bsp_cluster_portal_block, MAXIMUM_CLUSTER_PORTALS_PER_STRUCTURE, sizeof(struct cluster_portal))
{
	{_field_short_integer, "Back Cluster*", &structure_bsp_cluster_block},
	{_field_short_integer, "Front Cluster*", &structure_bsp_cluster_block},
	{_field_long_integer, "Plane Index*"},
	{_field_real_point_3d, "Centroid*"},
	{_field_real, "Bounding Radius*"},
	{_field_long_flags, "Flags*", "AI Cannot Hear through This:One-Way:Door:No-Way:One-Way Reversed:No One Can Hear through This"},

        .......
};
The Back/Front Cluster fields were defined the same way in Halo1. It's an oversight on whoever did the tag definitions for the sbsp. Either they used to be _block_index and where changed for whatever reason (to skip bounds checking on loading a corrupt bsp), or that was how they declared block indices in the very early days.
User avatar
kornman00
Posts: 104
Joined: Wed Jan 20, 2010 7:48 pm

Re: H2PC Guerilla tag layouts

Post by kornman00 »

JacksonCougar wrote:Good to know; xbox version is 3 bytes. Do you know if they changed the fields in any of the structs between xbox and PC?
Also, for PC they changed the animation_graph to keep its resources in the tag data, rather than offload them into the geometry cache (which is what the Xbox does, probably only for memory reasons). The PC's bitmap also added some fields related to WDP or w/e, but they're at the very end of the tag. Those are the only real changes I recall off hand to the tag definitions.
Post Reply