Porting Bungie's 3D Spasm Library to Three.js

Mar 22, 2018 | v1.0

Introduction

This article aims to explain in detail how Bungie's Spasm library works and how I developed a Three.js port of the code. There is a lot of information to cover and this article will require some revisions in the future, but I have written everything to the best of my knowledge and have indicated in places where I still lack information.

You can also check out the Gear Viewer page, which is a working demo of the code and an example of how it can be used.

Destiny Gear Viewer: A tool for searching and previewing 3d models from the games.

The full source code is available on github.


History

Along with the release of Destiny 1 back on September 9, 2014, Bungie developed a slew of other API-driven second screen resources that were intended to enhance the player experience. This included the Official Companion App for mobile devices as well as a dedicated character inspector on their website. On July 7, 2015 (more commonly known as Bungie Day) they also added an Armory which allowed players to search and inspect weapons and gear in the database. One thing all these tools had in common was that it was possible to preview items in 3d.

Bungie.net's Character Inspector that also let users manage their items. Bungie.net's Armory, which offered detailed information on items in the game.

This was achieved on Bungie.net through the use of a custom JavaScript library called Spasm. It handled everything from loading to rendering assets onto a HTML5 Canvas using WebGL and didn't rely on any third party libraries.

From my talks with vthornheart (one of the developers on the Bungie Web Team), Spasm was a pet project of one of the web developers who was working with a graphics engineer "on loan" to the Web Team. The loan was temporary and the web developer eventually joined the Mobile Team where he's continued to work on it, but with a focus now on getting it working on mobile rather than the website. When I followed up to check the details on this, vthornheart recalled there being a third guy from the mobile team who also assisted with the development. As he often tells folks before he pulls out a chair to explain why something is the way it is, "It's a long story..."

When Destiny 2 approached it's release date on September 6, 2017, they brought in engineers to update the renderer for use in the Official Destiny Companion app. There unfortunately wasn't enough time to bring forward many of the tools that were created for Destiny 1 on Bungie.net so all that is left now are screenshots and lots of dead links. The task of restoring and updating these resources falls on the third party developers within the Destiny community.

Unfortunately the Spasm library was not designed to be used anywhere other than Bungie.net and being proprietary meant adding new features would be difficult and cumbersome. Three.js is an open source library for rendering (and animating) 3d objects on the web. Not only is it still actively being maintained, but it requires very little 3d programming experience to create a scene and start drawing 3d objects to it.

I'd already tinkered with the library before while working on a Borderlands 2 character viewer project a few years back so it was a good opportunity to refresh myself on the library and have a go at porting Spasm to something that was more portable and most importantly, open source!


Pre-requisites

As all the gear asset data is served via the Bungie.net API, you'll need to create an Application and obtain an API Key to make requests to the API. You will also need to know how to download, extract and parse the Mobile Manifest which contains the item definitions among other things.

I highly recommend checking out the following guides before continuing:

The rest of this article will also refer to slides from past talks Bungie developers have presented at various conferences. These were key to understanding how all the data worked and offer some insight to how their game engine manages gear assets.


Content Urls

Below is a list of content urls that need to be prepended to any files referenced in the data. These urls are included with the manifest. They a relative to https://www.bungie.net.

"mobileGearCDN": {
    "Geometry": "/common/{game}_content/geometry/platform/{platform}/geometry",
    "Texture": "/common/{game}_content/geometry/platform/{platform}/textures",
    "PlateRegion": "/common/{game}_content/geometry/platform/{platform}/plated_textures",
    "Gear": "/common/{game}_content/geometry/gear",
    "Shader": "/common/{game}_content/geometry/platform/{platform}/shaders"
}

// game: destiny = Destiny 1, destiny2 = Destiny 2
// platform: mobile, web (Destiny 1 only)

Obtaining the Data

The first task in writing a port is being able to obtain the Gear Asset definition which contains the information needed to load an entire model. The companion app reads from the manifest while the website has a special endpoint that returns web friendly versions of textures. While porting over the code, I learned that the web versions were pre-baked plated textures which had been run through a compressor that indiscriminately resized them causing the image quality to vary between models. While the web version is more convenient, you will get noticeably better quality from using the mobile gear assets. Destiny 2 also only supports mobile assets.

Comparison of the texture quality between the mobile and web versions.

For the purposes of testing, I have created an endpoint on my website for querying the manifest. However if you plan on developing a fully fledged application, you should set up your own implementation. It's set up to match the response returned from the Destiny 1 endpoint that was used on Bungie.net.

// Destiny 1
// GET https://lowlidev.com.au/destiny/api/gearasset/{itemHash}?destiny

// Destiny 2
// GET https://lowlidev.com.au/destiny/api/gearasset/{itemHash}?destiny2

// Bungie.net Web Version Endpoint (Destiny 1 Only)
// GET https://www.bungie.net/d1/Platform/Destiny/Manifest/22/{itemHash}

For more information on the web version endpoint see Destiny.GetDestinySingleDefinition.


Gear Asset Definitions

Below is an example of what a gear asset definition looks like.

// Hash "3564229425" is the Gjallarhorn!!
// GET https://lowlidev.com.au/destiny/api/gearasset/3564229425?destiny
{
    // The requested itemHash
    "requestedId": "3564229425",

    // The actual gear asset definition
    "gearAsset": {
        // A json file containing art arrangement and dye settings, this is an array but there is only ever 1 entry
        "gear": [
            "18094812e64db9477a242d453002ee81.js"
        ],
        // All the asset content used by the item, again there is only every 1 entry
        "content": [
            {
                // The platform this content was made for, either "mobile" or "web"
                "platform": "mobile",
                // A list of files that contain geometry information
                "geometry": [
                    "6e8a8f1b9f97791be8708d25d2bba7bf.tgxm",
                    "4ef888dcaa9104965841b2b007b06df8.tgxm",
                    "ab8afb697fb3ef1c4188ae77a1b776b2.tgxm",
                    "d52efe8f2e6b44342950a2b80339d172.tgxm",
                    "dbdd156b7fd8adf104e4e955934ffcff.tgxm",
                    "afcefc510827c51ca9777a0c5a235b7e.tgxm"
                ],
                // A list of files that contain texture information
                // The web version returns pre-baked pngs/jpegs
                "textures": [
                    "38d13d99b099f3f660ae89bf2901eac6.tgxm.bin",
                    "50a9c256414f81311c307eaf057a96a8.tgxm.bin",
                    "a1eb2387ffff427e3c76f72be08b856b.tgxm.bin",
                    "2588e43d44f0c1c727253ad68191a8ba.tgxm.bin",
                    "d1d7ca22dff6a653443c1755eb638fb0.tgxm.bin",
                    "0898821c49b7b88ea380d344284e7fc7.tgxm.bin",
                    "bcb7fc29c4e457e680b099336f2368e3.tgxm.bin",
                    "749ebb80e4c3a2fd2223af93c6ba20b1.tgxm.bin",
                    "6a5d1e118bf3b2b261ca8981c6e935c4.tgxm.bin",
                    "41df9baf638cbb332d92785ada4f3be3.tgxm.bin",
                    "22ff9150b1f6f653c78f67470aa5091c.tgxm.bin",
                    "359283780a11da557ccffb1ed9fda81c.tgxm.bin",
                    "893ee480a2220af5913178270c7f99d4.tgxm.bin",
                    "dfde534f31dc448d19426f88f2edf293.tgxm.bin",
                    "d1821088dda6df9461b4da3fdef14027.tgxm.bin",
                    "4e8326c7c91cb35e90a8906487d4356b.tgxm.bin"
                ],
                // A lookup table of assets used for dyes
                "dye_index_set": {
                    "textures": [
                        0,
                        1,
                        2,
                        3,
                        4,
                        5
                    ],
                    "geometry": []
                },
                // A lookup table of assets used for each region
                "region_index_sets": {
                    "1": [
                        {
                            "textures": [
                                6,
                                7
                            ],
                            "geometry": [
                                0
                            ]
                        }
                    ],
                    "0": [
                        {
                            "textures": [
                                8
                            ],
                            "geometry": [
                                1
                            ]
                        }
                    ],
                    "5": [
                        {
                            "textures": [
                                9
                            ],
                            "geometry": [
                                2
                            ]
                        }
                    ],
                    "3": [
                        {
                            "textures": [
                                10
                            ],
                            "geometry": [
                                3
                            ]
                        }
                    ],
                    "4": [
                        {
                            "textures": [
                                11,
                                12,
                                13,
                                14
                            ],
                            "geometry": [
                                4
                            ]
                        }
                    ],
                    "21": [
                        {
                            "textures": [
                                15
                            ],
                            "geometry": [
                                5
                            ]
                        }
                    ]
                }
            }
        ]
    }
}

There's some terminology in there that won't make sense unless you understand how assets are actually loaded in Destiny. What you need to understand here is that this definition contains data for every variation the item can have, whether it's gendered body parts or different scopes on a sniper rifle. This allows you to optimize your loader so it only loads what assets are actually used.


TGX Container Format

Most of the actual geometry and texture data is stored within what is known as a TGX container file.

struct tgxEntry {
    string[256] name;
    uint offset; // relative to the start of the file
    uint type; // always 0
    uint size;
}
struct tgxFormat {
    string[4] magic; // spells "TGXM"
    uint version;
    uint fileHeaderOffset;
    uint fileCount;
    string[256] fileIdentifier; // Always in the format XXXXXXXXXX-X
    tgxEntry[fileCount] entries;
    byte[] data; // Raw file data
}

Geometry containers will always contain the following files:

  • 0.0.vertexbuffer.tgx // Vertex position info
  • 0.1.vertexbuffer.tgx // Texcoord and other vertex info
  • 0.indexbuffer.tgx
  • render_metadata.js // JSON metadata file

Texture containers hold grouped texture maps, ie diffuse, normal, gearstack maps. This varies based on the shader used.


TGX Render Metadata

The render metadata file describes how to parse the tgx files which contain the raw geometry data.

"render_model": {
    // A list of meshes used in this model, always should be 1
    "render_meshes": [
        {
            // The bounding box of this mesh
            "bounding_volume": {
                "max_x": 0.396251,
                "max_y": 0.07521442,
                "max_z": 0.230731428,
                "min_x": -0.279061,
                "min_": -0.07521442,
                "min_z": -0.108645432
            },

            // Describes the file in the tgx container which holds the index buffer
            "index_buffer": {
                "file_name": "0.indexbuffer.tgx",
                "byte_size": 9212,
                "value_byte_size": 2
            },

            // Vertex scale and positioning
            "position_offset": [0.058595, 0, 0.061043],
            "position_scale": [0.337656, 0.337656, 0.337656],

            // A list of parts that make up this mesh
            "stage_part_list": [
                {
                    "external_identifier": 0,
                    "flags": 5,

                    // References a lookup table that describes how a gear dye is applied
                    "gear_dye_change_color_index": 0,

                    // The size of the part index buffer (relative to start_indeX)
                    "index_count": 225,
                    "index_max": 117,
                    "index_min": 0,
                    "lod_category": {
                        "value": 8,
                        "name": "_lod_category_23"
                    },
                    "lod_run": 1,

                    // 3=triangles, 5=triangle strip
                    "primitive_type": 5,

                    // Information about the shader used for this part
                    "shader": {
                        // Common static textures that are shared across all models
                        "static_textures": [
                            "2164797681_default_monocrome_cubemap"
                        ],
                        "type": 9
                    },

                    // The offset into the index buffer where this part starts
                    "start_index": 1,
                    "variant_shader_index": -1
                },
                ...
            ],
            "stage_part_offsets": [0, 3, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, 9, 9, 9, 9, 9, 9, 9],

            // Describes how to unpack vertex buffer data, ie position, normal, tangent, texcoord
            // There's only ever 1 layout definition
            "stage_part_vertex_stream_layout_definitions": [
                {
                    "formats": [
                        {
                            "elements": [
                                {
                                    "normalized": false,
                                    "offset": 0,
                                    "semantic": "_tfx_vb_semantic_position",
                                    "semantic_index": 0,
                                    "size": 16,
                                    "type": "_vertex_format_attribute_float4"
                                }
                            ],
                            "stride": 16
                        },
                        ...
                    ],
                    "type": "_vertex_stream_layout_ao_precise_optimized_threshold"
                }
            ],
            "stage_part_vertex_stream_layout_lookup": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

            // UV map scale and positioning
            "texcoord0_scale_offset": [0.373157, 0.373157, 0.374901, 0.374659]
            "texcoord_offset": [0.374901, 0.374659]
            "texcoord_scale": [0.373157, 0.373157]

            // Describes the files in the tgx container which hold vertex buffer data
            "vertex_buffers": [
                {
                    "file_name": "0.0.vertexbuffer.tgx",
                    "byte_size": 34944,
                    "stride_byte_size": 16
                },
                {
                    "file_name": "0.1.vertexbuffer.tgx",
                    "byte_size": 61152,
                    "stride_byte_size": 28
                }
            ]
        }
    ],
    // Describes the texture plates used by this model
    "texture_plates": [
        {
            "gear_decal_dye_index": -1,
            "gear_slot_requires_plating": true,
            "number_of_gear_slots": 0,
            "number_of_plateable_gear_slots": 0

            // A plate set is made up of 3 texture maps, diffuse, normal and gearstack
            "plate_set": {
                "diffuse": {
                    "reference_id": "4a18ada4ebb4d9d46dc7d3fada96b3cf",
                    "plate_index": 0,
                    "plate_size": [512, 512],
                    "texture_placements": [
                        {
                            "texture_size_x": 384,
                            "texture_size_y": 256,
                            "position_x": 0,
                            "position_y": 0,
                            "texture_tag_name": "2445676024_exotic_15_frame_gbit_384_256_0"
                        }
                    ]
                },
                "gearstack": {
                    "reference_id": "428f7afa0ce12186f616cf5eb98dd5e1",
                    "plate_index": 0,
                    "plate_size": [512, 512],
                    "texture_placements": [
                        {
                            "texture_size_x": 384,
                            "texture_size_y": 256,
                            "position_x": 0,
                            "position_y": 0,
                            "texture_tag_name": "2445676024_exotic_15_frame_gbit_384_256_2"
                        }
                    ]
                }
                "normal": {
                    "reference_id": "9abc01c6ad2822ab70b2a1e85ba3d075",
                    "plate_index": 0,
                    "plate_size": [512, 512],
                    "texture_placements": [
                        {
                            "texture_size_x": 384,
                            "texture_size_y": 256,
                            "position_x": 0,
                            "position_y": 0,
                            "texture_tag_name": "2445676024_exotic_15_frame_gbit_384_256_1"
                        }
                    ]
                }
            }
        }
    ]
}

Vertex Buffers

Each vertex buffer entry has a corresponding format in stage_part_vertex_stream_layout_definitions. The format describes how the vertex data is packed.

{
    // Describes how to unpack vertex buffer data, ie position, normal, tangent, texcoord
    // There's only every 1 layout definition
    "stage_part_vertex_stream_layout_definitions": [
        {
            "formats": [
                {
                    "elements": [
                        {
                            "normalized": false,
                            "offset": 0,
                            "semantic": "_tfx_vb_semantic_position",
                            "semantic_index": 0,
                            "size": 16,
                            "type": "_vertex_format_attribute_float4"
                        }
                    ],
                    "stride": 16
                },
                ...
            ],
            "type": "_vertex_stream_layout_ao_precise_optimized_threshold"
        }
    ],

    // Describes the files in the tgx container which hold vertex buffer data
    "vertex_buffers": [
        {
            "file_name": "0.0.vertexbuffer.tgx",
            "byte_size": 34944,
            "stride_byte_size": 16
        },
        {
            "file_name": "0.1.vertexbuffer.tgx",
            "byte_size": 61152,
            "stride_byte_size": 28
        }
    ]
}

The type property describes the datatype. It's in the format _vertex_format_attribute_{datatype}{bytesize} and there are 7 datatypes.

  • ubyte
  • byte
  • short
  • ushort
  • int
  • uint
  • float

If normalized is true it means the value needs to be converted to a floating point number based on the number of bits. See Normal number (computing).

The semantic describes what the element is for. There are 7 semantic types.

  • position
  • normal
  • tangent
  • texcoord
  • blendweight
  • blendindices
  • color

The semantic_index is used when there are multiple semantic elements in the same format, ie multiple texcoords.


Stage Parts

Stage parts are used to determine what geometry to display based on how far away the object is from the camera. Unfortunately it's not clear how this is done as the Spasm library hard codes it, excluding things like scope rectiles and decals. I've filed an issue on the Bungie.net API github to hopefully get more information so I can add support for other shaders that the original Spasm library never implemented.

{
    "external_identifier": 0,
    "flags": 5,

    // References a lookup table that describes how a gear dye is applied
    "gear_dye_change_color_index": 0,

    // The size of the part index buffer (relative to start_indeX)
    "index_count": 225,
    "index_max": 117,
    "index_min": 0,
    "lod_category": {
        "value": 8,
        "name": "_lod_category_23"
    },
    "lod_run": 1,

    // 3=triangles, 5=triangle strip
    "primitive_type": 5,

    // Information about the shader used for this part
    "shader": {
        // Common static textures that are shared across all models
        "static_textures": [
            "2164797681_default_monocrome_cubemap"
        ],
        "type": 9
    },

    // The offset into the index buffer where this part starts
    "start_index": 1,
    "variant_shader_index": -1
}

The gear_dye_change_color_index uses the following lookup table:

Change Color Index Gear Dye Slot Use Primary Color Use Investment Decal
0 0 true false
1 0 false false
2 1 true false
3 1 false false
4 2 true false
5 2 false false
6 3 true true
7 3 true true
Siggraph 2014 - Dye Slot Setup For Gear Arrangements

Texture Plates

Gear assets in Destiny use a system where textures are baked into plate textures. The texture used for a given piece of geometry will always occupy the same region on a plate.

{
    "gear_decal_dye_index": -1,
    "gear_slot_requires_plating": true,
    "number_of_gear_slots": 0,
    "number_of_plateable_gear_slots": 0

    // A plate set is made up of 3 texture maps, diffuse, normal and gearstack
    "plate_set": {
        "diffuse": {
            "reference_id": "4a18ada4ebb4d9d46dc7d3fada96b3cf",
            "plate_index": 0,
            "plate_size": [512, 512],
            "texture_placements": [
                {
                    "texture_size_x": 384,
                    "texture_size_y": 256,
                    "position_x": 0,
                    "position_y": 0,
                    "texture_tag_name": "2445676024_exotic_15_frame_gbit_384_256_0"
                }
            ]
        },
        "gearstack": {
            "reference_id": "428f7afa0ce12186f616cf5eb98dd5e1",
            "plate_index": 0,
            "plate_size": [512, 512],
            "texture_placements": [
                {
                    "texture_size_x": 384,
                    "texture_size_y": 256,
                    "position_x": 0,
                    "position_y": 0,
                    "texture_tag_name": "2445676024_exotic_15_frame_gbit_384_256_2"
                }
            ]
        }
        "normal": {
            "reference_id": "9abc01c6ad2822ab70b2a1e85ba3d075",
            "plate_index": 0,
            "plate_size": [512, 512],
            "texture_placements": [
                {
                    "texture_size_x": 384,
                    "texture_size_y": 256,
                    "position_x": 0,
                    "position_y": 0,
                    "texture_tag_name": "2445676024_exotic_15_frame_gbit_384_256_1"
                }
            ]
        }
    }
}
Sample plated diffuse map

Each plate can have multiple texture map sets, most often a diffuse, normal and gearstack map.

Plate Set for the "Celestial Nighthawk" Hunter Helmet

Restricting texture placements to pre-defined regions allows the renderer to combine them into larger plate textures.

Siggraph 2014 - Plate Set - Warlock

The pre-baked plated textures in the web version did not actually implement plated textures correctly, instead creating a separate plated size texture for each sub texture.


The Gearstack

A common technique used in game engines is to have a masking map which holds different data in each color channel. For Destiny, this is the gearstack map.

While tweeting about the gearstack, I actually got some input from Graphics Technical Art Lead, Nate Hawbaker who explained what each color channel was used for in Destiny 2. I also had to go look up half the words he tweeted after to understand what he was saying! >_<

  • red: ambient occlusion
  • green: smoothness
  • blue: encoded alpha test and emissive
  • alpha: encoded dye mask, non-dyed metalness and wear mask
Destiny 2 gearstack split out into each channel.

Vivek Hari also provided some information on the Destiny 1 gearstack.

  • red: scratch mask
  • green: specular roughness
  • blue: varied, ie fringe maps & alpha blending
Destiny 1 gearstack of the same model.

While I do have this information, I still have the problem of not knowing when to use it since there is a lot of flags and stuff in the data I don't yet understand.


Gear Dyes and Art Arrangements

The json file referenced in the gear asset definition (which I will refer to as gear.js) has information about the dyes and different art arrangements it can have.

// GET https://www.bungie.net/common/destiny_content/geometry/gear/ddd0254bafd151245b184577f55c9a66.js
{
  "default_dyes": [],
  "locked_dyes": [
    {
      "hash": 4052828031,
      "investment_hash": 2672455039,
      "slot_type_index": 0,
      "variant": 0,
      "blend_mode": 0,
      "cloth": false,
      "material_properties": {
        "primary_color": [
          0.501961,
          0.501961,
          0.501961,
          1.0
        ],
        "secondary_color": [
          0.154549,
          0.154549,
          0.154549,
          1.0
        ],
        "detail_transform": [
          4.0,
          4.0,
          0.0,
          0.0
        ],
        "detail_normal_contribution_strength": [
          0.5,
          0.5,
          0.5,
          0.5
        ],
        "decal_alpha_map_transform": [
          1.0,
          1.0,
          0.0,
          0.0
        ],
        "decal_blend_option": 1,
        "specular_properties": [
          0.00098,
          0.0,
          1.0,
          -0.2
        ],
        "subsurface_scattering_strength": [
          32.3,
          1.0,
          1.0,
          1.0
        ]
      },
      "textures": {
        "diffuse": {
          "name": "2164797681_128_grey_128_alpha_linear_dif",
          "reference_id": "dfa5a1a0313319f22ca0ce376416d1bf"
        },
        "normal": {
          "name": "2164797681_default_normal",
          "reference_id": "b55c062f45e1082ea2f493e9f8181690"
        },
        "decal": {
          "name": ""
        },
        "primary_diffuse": {
          "name": "_808080FF_2164797681_128_grey_128_alpha_linear_dif",
          "reference_id": "d75e39895c5a0458803c94cd6b1b423d"
        },
        "secondary_diffuse": {
          "name": "_272727FF_2164797681_128_grey_128_alpha_linear_dif",
          "reference_id": "f8166efa73eca0b2048e29851ee104bf"
        }
      }
    }
    ...
  ],
  "custom_dyes": [],
  "reference_id": "1274330687",
  "art_content": {
    "hash": 1274330687,
    "gear_set": {
      "regions": [
        {
          "region_index": 1,
          "pattern_list": [
            {
              "hash": 3152100547,
              "geometry_hashes": [
                "3152100547-0"
              ]
            }
          ]
        }
        ...
      ],
      "base_art_arrangement": {
        "hash": 2166136261
      },
      "female_override_art_arrangement": {
        "hash": 2166136261
      }
    }
  },
  "art_content_sets": [
    {
      "classHash": 0,
      "arrangement": {
        "hash": 1274330687,
        "gear_set": {
          "regions": [
            {
              "region_index": 1,
              "pattern_list": [
                {
                  "hash": 3152100547,
                  "geometry_hashes": [
                    "3152100547-0"
                  ]
                }
              ]
            }
            ...
          ],
          "base_art_arrangement": {
            "hash": 2166136261
          },
          "female_override_art_arrangement": {
            "hash": 2166136261
          }
        }
      }
    }
  ]
}

Gear Dyes

Gear dyes are how the renderer applies color to textures, not only for shaders but for applying default colors as well.

{
    "default_dyes": [],
    "locked_dyes": [],
    "custom_dyes": []
}

An example of the dye data structure.

{
  "hash": 4052828031,
  "investment_hash": 2672455039,
  "slot_type_index": 0,
  "variant": 0,
  "blend_mode": 0,
  "cloth": false,
  "material_properties": {
    "primary_color": [
      0.501961,
      0.501961,
      0.501961,
      1.0
    ],
    "secondary_color": [
      0.154549,
      0.154549,
      0.154549,
      1.0
    ],
    "detail_transform": [
      4.0,
      4.0,
      0.0,
      0.0
    ],
    "detail_normal_contribution_strength": [
      0.5,
      0.5,
      0.5,
      0.5
    ],
    "decal_alpha_map_transform": [
      1.0,
      1.0,
      0.0,
      0.0
    ],
    "decal_blend_option": 1,
    "specular_properties": [
      0.00098,
      0.0,
      1.0,
      -0.2
    ],
    "subsurface_scattering_strength": [
      32.3,
      1.0,
      1.0,
      1.0
    ]
  },
  "textures": {
    "diffuse": {
      "name": "2164797681_128_grey_128_alpha_linear_dif",
      "reference_id": "dfa5a1a0313319f22ca0ce376416d1bf"
    },
    "normal": {
      "name": "2164797681_default_normal",
      "reference_id": "b55c062f45e1082ea2f493e9f8181690"
    },
    "decal": {
      "name": ""
    },
    "primary_diffuse": {
      "name": "_808080FF_2164797681_128_grey_128_alpha_linear_dif",
      "reference_id": "d75e39895c5a0458803c94cd6b1b423d"
    },
    "secondary_diffuse": {
      "name": "_272727FF_2164797681_128_grey_128_alpha_linear_dif",
      "reference_id": "f8166efa73eca0b2048e29851ee104bf"
    }
  }
}

There are 3 gear dye types:

  • Default Dyes
  • Custom Dyes (for applying shaders)
  • Locked Dyes (often found on exotic items or regions that will ignore shaders)

When determining which dye to use, the renderer will prioritize locked dyes first, custom dyes second and will fallback to default dyes.

Vex Mythoclast before and after applying gear dyes.

In Destiny 2, the rendering changed to use Physically Based Rendering, this means the dye data structure has changed too.

{
  "hash": 2378398189,
  "investment_hash": 3504223213,
  "slot_type_index": 0,
  "cloth": false,
  "material_properties": {
    "detail_diffuse_transform": [
      7.5,
      7.5,
      0.0,
      0.0
    ],
    "detail_normal_transform": [
      7.5,
      7.5,
      0.0,
      0.0
    ],
    "spec_aa_xform": [
      1.992188,
      -1.0,
      0.4,
      0.0
    ],
    "emissive_tint_color_and_intensity_bias": [
      1.0,
      0.179926,
      0.049707,
      0.778644
    ],
    "specular_properties": [
      0.00098,
      0.0,
      1.0,
      0.0
    ],
    "lobe_pbr_params": [
      0.55,
      0.3,
      0.921954,
      0.0
    ],
    "tint_pbr_params": [
      0.187256,
      0.0,
      0.0,
      1.2304
    ],
    "emissive_pbr_params": [
      0.0,
      0.0,
      0.0,
      0.0
    ],
    "primary_albedo_tint": [
      0.995018,
      0.740509,
      0.208356,
      1.0
    ],
    "primary_material_params": [
      0.0,
      0.0,
      0.0,
      1.0
    ],
    "primary_roughness_remap": [
      -4.933334,
      6.666668,
      0.74,
      0.15
    ],
    "secondary_albedo_tint": [
      0.877438,
      0.90782,
      0.90782,
      1.0
    ],
    "secondary_material_params": [
      0.0,
      0.0,
      0.0,
      1.0
    ],
    "secondary_roughness_remap": [
      2.022222,
      6.31111,
      0.25,
      0.09
    ],
    "worn_albedo_tint": [
      0.552011,
      0.571125,
      0.571125,
      1.0
    ],
    "wear_remap": [
      -3.406,
      4.926,
      0.0,
      1.0
    ],
    "worn_roughness_remap": [
      -1.273333,
      3.006667,
      0.74,
      0.15
    ],
    "worn_material_parameters": [
      0.0,
      0.0,
      1.0,
      1.0
    ],
    "subsurface_scattering_strength_and_emissive": [
      32.3,
      1.0,
      8.705548,
      0.0
    ]
  },
  "textures": {
    "diffuse": {
      "name": "3104558709_Kevlar_dif",
      "reference_id": "e08f46e2484cbca4493ca78c2a9768d9"
    },
    "normal": {
      "name": "3104558709_Kevlar_norm",
      "reference_id": "9d7b45d880bee7e0e1ef67186575058a"
    },
    "primary_diffuse": {
      "name": "_FFFFFFFF_3104558709_Kevlar_dif",
      "reference_id": "e08f46e2484cbca4493ca78c2a9768d9"
    },
    "secondary_diffuse": {
      "name": "_FFFFFFFF_3104558709_Kevlar_dif",
      "reference_id": "e08f46e2484cbca4493ca78c2a9768d9"
    }
  }
}

Art Arrangements

Gear can have one or more art arrangements. If art_content_sets is present, the render will need to determine which arrangement is appropriate by checking the classHash value.

{
    "art_content": {},
    "art_content_sets": [
        {
            "classHash": 0,
            "arrangement": {}
        }
    ]
}

An example of the art arrangement data structure.

{
    "hash": 1274330687,
    "gear_set": {
      "regions": [
        {
          "region_index": 1,
          "pattern_list": [
            {
              "hash": 3152100547,
              "geometry_hashes": [
                "3152100547-0"
              ]
            }
          ]
        }
        ...
      ],
      // The default art arrangement
      "base_art_arrangement": {
        "hash": 2166136261
      },
      // If this arrangement is present, the renderer can choose to display female variations of the geometry instead
      "female_override_art_arrangement": {
        "hash": 2166136261
      }
    }
}

Loading and Parsing the Data

There is a number of steps to take an itemHash (and optionally a shaderItemHash) and obtain the data we need to load it into Three.js.

Below is a rough overview:

  • Request gear asset definition from the manifest
  • Load the gear.js file
  • Load each tgx container file and extract the geometry and textures into lookup tables
  • Determine which regions will be rendered based on class and gender
  • Combine item and shader gear dyes (if any) and load them into the gear dye slots
  • Build plate sets by stitching textures together
  • Determine which stage parts should be rendered and parse the geometry data into the buffer

Building An Asset Loader

A single gear asset can have upwards of 20 different asset files, some which may also contain references to other files, so you need to have a decent load manager to keep track of all these files and make sure everything is loaded before you starting parsing them.

var gear;
var itemHash = 1274330687;
var isFemale = false;

// Request gear asset definition from the manifest
// GET https://lowlidev.com.au/destiny/api/gearasset/3564229425?destiny

loadAssetManifest(gear);

function loadAssetManifest(gear) {
    var i, j;

    var requestedId = gear.requestedId;
    var gearAsset = gear.gearAsset;

    for (i=0; i<gearAsset.content.length; i++) {
        var content = gearAsset.content[i];
        loadAssetContent(content);
    }

    for (i=0; i<gearAsset.gear.length; i++) {
        var gearFilename = gearAsset.gear[i];
        // GET https://www.bungie.net/common/{game}_content/geometry/gear/{gearFilename}
    }
}

function loadAssetContent(content) {
    var filteredRegionIndexSets = [];

    if (content.dye_index_set) {
        filteredRegionIndexSets.push(content.dye_index_set);
    }

    if (content.region_index_sets) { // Use gender neutral sets
        for (var setIndex in content.region_index_sets) {
            var regionIndexSet = content.region_index_sets[setIndex];

            for (j=0; j<regionIndexSet.length; j++) {
                filteredRegionIndexSets.push(regionIndexSet[j]);
            }
        }
    }
    else if (content.female_index_set && content.male_index_set) { // Use gender-specific set (ie armor)
        filteredRegionIndexSets.push(isFemale ? content.female_index_set : content.male_index_set);
    }

    // Build Asset Index Table
    var geometryIndexes = {};
    var textureIndexes = {};
    var platedTextureIndexes = {};

    for (var filteredRegionIndex in filteredRegionIndexSets) {
        var filteredRegionIndexSet = filteredRegionIndexSets[filteredRegionIndex];
        var index, i;
        if (filteredRegionIndexSet == undefined) {
            console.warn('MissingFilterRegionIndexSet', filteredRegionIndex, filteredRegionIndexSets);
            continue;
        }
        // Shaders don't have geometry
        if (filteredRegionIndexSet.geometry) {
            for (i=0; i<filteredRegionIndexSet.geometry.length; i++) {
                index = filteredRegionIndexSet.geometry[i];
                geometryIndexes[index] = index;
            }
        }
        for (i=0; i<filteredRegionIndexSet.textures.length; i++) {
            index = filteredRegionIndexSet.textures[i];
            textureIndexes[index] = index;
        }
        // Web only
        if (filteredRegionIndexSet.plate_regions) {
            for (i=0; i<filteredRegionIndexSet.plate_regions.length; i++) {
                index = filteredRegionIndexSet.plate_regions[i];
                platedTextureIndexes[index] = index;
            }
        }
        // Apparently there are shaders?
        if (filteredRegionIndexSet.shaders) {
            console.warn('AssetHasShaders['+i+']', filteredRegionIndexSet.shaders);
        }
    }

    // Load Geometry
    for (var geometryIndex in geometryIndexes) {
        var geometryFilename = content.geometry[geometryIndex];
        // GET https://www.bungie.net/common/{game}_content/geometry/platform/{platform}/geometry/{geometryFilename}
        // loadTGXBin(url);
    }

    // Load Textures
    for (var textureIndex in textureIndexes) {
        var textureFilename = content.textures[textureIndex];
        // GET https://www.bungie.net/common/{game}_content/geometry/platform/{platform}/textures/{textureFilename}
        // loadTexture(url);
    }

    // Load Plated Textures (web version only)
    for (var platedTextureIndex in platedTextureIndexes) {
        var platedTextureFilename = content.plate_regions[platedTextureIndex];
        // GET https://www.bungie.net/common/{game}_content/geometry/platform/{platform}/textures/{platedTextureFilename}
        // loadTexture(url);
    }
}

Here's an example of loading a TGX container file. Note, "utils" is a set of helper functions for manipulating binary data.

function loadTGXBin(url) {
    var data; // GET url

    var magic = utils.string(data, 0x0, 0x4); // TGXM
    var version = utils.uint(data, 0x4);
    var fileOffset = utils.uint(data, 0x8);
    var fileCount = utils.uint(data, 0xC);
    var fileIdentifier = utils.string(data, 0x10, 0x100);

    if (magic != 'TGXM') {
        console.error('Invalid TGX File', url);
        return;
    }
    var files = [];
    var fileLookup = [];
    var renderMetadata = false;

    // Iterate through the files in the container. We need a lookup table of filenames for matching references in the render_metadata file.
    for (var f=0; f<fileCount; f++) {
        var headerOffset = fileOffset+0x110*f;
        var name = utils.string(data, headerOffset, 0x100);
        var offset = utils.uint(data, headerOffset+0x100);
        var type = utils.uint(data, headerOffset+0x104);
        var size = utils.uint(data, headerOffset+0x108);
        var fileData = data.slice(offset, offset+size);

        if (name.indexOf('.js') != -1) { // render_metadata.js
            fileData = JSON.parse(utils.string(fileData));
            renderMetadata = fileData;
        }
        files.push({
            name: name,
            offset: offset,
            type: type,
            size: size,
            data: fileData
        });
        fileLookup.push(name);
    }

    return {
        url: url,
        fileIdentifier: fileIdentifier,
        files: files,
        lookup: fileLookup,
        metadata: renderMetadata
    };
}

For textures, we need to check to see whether the referenced files are TGX containers or plated pngs/jpegs. Take note of the filename and referenceId as these both may get used to reference textures in other places.

function loadTexture(url) {
    if (url.indexOf('.bin') != -1) { // Mobile texture
        var tgxBin = loadTGXBin(url);
        for (var i=0; i<tgxBin.files.length; i++) {
            var file = tgxBin.files[i];
            // Add texture to lookup table
        }
    } else {
        // Treat file as an png/jpeg image and add it to the lookup table
    }
}

Parsing Gear Data

Once all the referenced asset files have been loaded you'll have something that looks like this.

// Loaded content for itemHash "3564229425", aka our Gjallarhorn
var contentLoaded = {
    "items": [
        {
            "requestedId": "3564229425",
            // GET https://lowlidev.com.au/destiny/api/gearasset/3564229425?destiny
            "gearAsset": {
                ...
            },
            "shaderHash": 0, // No shader specified
        }
    ],
    // Loaded gear.js file
    "gear": {
        // GET https://www.bungie.net/common/destiny_content/geometry/gear/ddd0254bafd151245b184577f55c9a66.js
        "3564229425": {
            "default_dyes": [],
            "locked_dyes": [
                ...
            ],
            "custom_dyes": [],
            "reference_id": "3564229425",
            "art_content": {
                ...
            },
            "art_content_sets": [
                ...
            ]
        }
    },

    // Loaded geometry files
    "geometry": {
        "1366948101-0": {}, // d52efe8f2e6b44342950a2b80339d172.tgxm
        "1606185591-0": {}, // dbdd156b7fd8adf104e4e955934ffcff.tgxm
        "2309204541-0": {}, // ab8afb697fb3ef1c4188ae77a1b776b2.tgxm
        "3152100547-0": {}, // 6e8a8f1b9f97791be8708d25d2bba7bf.tgxm
        "3239100822-0": {}, // 4ef888dcaa9104965841b2b007b06df8.tgxm
        "3801505009-0": {} // afcefc510827c51ca9777a0c5a235b7e.tgxm
    }
    // Loaded texture files that have been unpacked from TGX container files
    "textures": {
        // 4e8326c7c91cb35e90a8906487d4356b.tgxm.bin
        "633833313_fotc_target_reticles_0": {},
        "633833313_fotc_target_reticles_1": {},
        "633833313_fotc_target_reticles_2": {},
        "633833313_fotc_target_reticles_4": {},
        "633833313_fotc_target_reticles_5": {},
        "633833313_fotc_target_reticles_6": {},
        "633833313_fotc_target_reticles_7": {},

        // 4e8326c7c91cb35e90a8906487d4356b.tgxm.bin
        "633833313_fotc_target_reticles_hecate": {},

        // 893ee480a2220af5913178270c7f99d4.tgxm.bin
        "978348035_exotic_15_stock_gbit_384_192_0": {},
        "978348035_exotic_15_stock_gbit_384_192_1": {},
        "978348035_exotic_15_stock_gbit_384_192_2": {},

        // bcb7fc29c4e457e680b099336f2368e3.tgxm.bin
        "1249341125_exotic_15_barrel_gbit_128_320_0": {},
        "1249341125_exotic_15_barrel_gbit_128_320_1": {},
        "1249341125_exotic_15_barrel_gbit_128_320_2": {},

        // dfde534f31dc448d19426f88f2edf293.tgxm.bin
        "1735486583_fotc_decals_common_diffuse": {},

        // 38d13d99b099f3f660ae89bf2901eac6.tgxm.bin
        "2164797681_128_grey_128_alpha_linear_dif": {},

        // 2588e43d44f0c1c727253ad68191a8ba.tgxm.bin
        "2164797681_default_normal": {},

        // 749ebb80e4c3a2fd2223af93c6ba20b1.tgxm.bin
        "2293609183_exotic_15_hatch_gbit_128_128_0": {},
        "2293609183_exotic_15_hatch_gbit_128_128_0": {},
        "2293609183_exotic_15_hatch_gbit_128_128_1": {},
        "2293609183_exotic_15_hatch_gbit_128_128_2": {},

        // 359283780a11da557ccffb1ed9fda81c.tgxm.bin
        "2339811273_exotic_15_scope_tubes_gbit_384_64_0": {},
        "2339811273_exotic_15_scope_tubes_gbit_384_64_1": {},
        "2339811273_exotic_15_scope_tubes_gbit_384_64_2": {},

        // 6a5d1e118bf3b2b261ca8981c6e935c4.tgxm.bin
        "2445676024_exotic_15_frame_gbit_384_256_0": {},
        "2445676024_exotic_15_frame_gbit_384_256_1": {},
        "2445676024_exotic_15_frame_gbit_384_256_2": {},

        // 22ff9150b1f6f653c78f67470aa5091c.tgxm.bin
        "2556277053_exotic_15_scope_gbit_256_256_0": {},
        "2556277053_exotic_15_scope_gbit_256_256_1": {},
        "2556277053_exotic_15_scope_gbit_256_256_2": {},

        // d1821088dda6df9461b4da3fdef14027.tgxm.bin
        "2591025506_scope_glass_cubemap_env": {},

        // 41df9baf638cbb332d92785ada4f3be3.tgxm.bin
        "3519070587_exotic_15_ammo_gbit_128_64_0": {},
        "3519070587_exotic_15_ammo_gbit_128_64_1": {},
        "3519070587_exotic_15_ammo_gbit_128_64_2": {},

        // d1d7ca22dff6a653443c1755eb638fb0.tgxm.bin
        "_222FF_2164797681_128_grey_128_alpha_linear_dif": {},

        // 0898821c49b7b88ea380d344284e7fc7.tgxm.bin
        "_999FF_2164797681_128_grey_128_alpha_linear_dif": {},

        // a1eb2387ffff427e3c76f72be08b856b.tgxm.bin
        "_272727FF_2164797681_128_grey_128_alpha_linear_dif": {},

        // 50a9c256414f81311c307eaf057a96a8.tgxm.bin
        "_808080FF_2164797681_128_grey_128_alpha_linear_dif": {}
    }
};

The first step is to figure out what art regions need to be rendered.

function parseItem(item) {
    var gear = contentLoaded.gear[item.requestedId];
    var shaderGear = item.shaderHash ? contentLoaded.gear[item.shaderHash] : null;

    // Note loading a shader is the same process as loading an item, the only difference is that shaders won't contain any geometry

    // TODO: Should iterate this, but its never has more than one
    var regionIndexSets = item.gearAsset.content[0].region_index_sets;
    var assetIndexSet = contentLoaded.regions[item.requestedId]; // contentLoaded is where we stored our loaded files

    // Figure out which geometry should be loaded ie class, gender
    var artContent = gear.art_content;
    var artContentSets = gear.art_content_sets;
    if (artContentSets && artContentSets.length > 1) {
        //console.log('Requires Arrangement', artContentSets);
        for (var r=0; r<artContentSets.length; r++) {
            var artContentSet = artContentSets[r];
            if (artContentSet.classHash == classHash) {
                artContent = artContentSet.arrangement;
                break;
            }
        }
    } else if (artContentSets && artContentSets.length > 0) {
        artContent = artContentSets[0].arrangement;
    }

    var artRegionPatterns = [];

    if (artContent) {
        var gearSet = artContent.gear_set;
        var regions = gearSet.regions;
        if (regions.length > 0) {
            for (var u=0; u<regions.length; u++) {
                var region = regions[u];
                var regionIndexSet = regionIndexSets[region.region_index];
                //console.log('Region['+u+':'+region.region_index+']', region.pattern_list, regionIndexSet);

                if (region.pattern_list.length > 1) {
                    console.warn('MultiPatternRegion['+u+']', region);
                    // Weapon attachments?
                }
                for (var p=0; p<region.pattern_list.length; p++) {
                    var pattern = region.pattern_list[p];
                    //var patternIndex = regionIndexSet[p];

                    artRegionPatterns.push({
                        hash: pattern.hash,
                        artRegion: u,
                        patternIndex: p,
                        regionIndex: region.region_index,
                        geometry: pattern.geometry_hashes
                    });

                    //
                    break;
                }
            }
        } else {
            var overrideArtArrangement = isFemale ? gearSet.female_override_art_arrangement : gearSet.base_art_arrangement;
            artRegionPatterns.push({
                hash: overrideArtArrangement.hash,
                artRegion: isFemale ? 'female' : 'male',
                patternIndex: -1,
                regionIndex: -1,
                geometry: overrideArtArrangement.geometry_hashes
            });
        }
    }

    // Parse Gear Dyes
    var gearDyes = parseGearDyes(gear, shaderGear);

    // Parse Art Regions
    parseArtRegions(artRegionPatterns, gearDyes);
}

Parsing Gear Dyes

Gear dyes are at the heart of gear rendering in Destiny. By including a shaderGear, you can effectively swap out dye information and customize the look of a piece of gear without needing to create new assets for every item.

function parseGearDyes(gear, shaderGear) {
    var gearDyeGroups = getGearDyes(gear);
    var shaderDyeGroups = shaderGear ? getGearDyes(shaderGear) : gearDyeGroups;

    // Spasm.GearRenderable.prototype.getResolvedDyeList
    var resolvedDyes = [];
    var dyeTypeOrder = ['defaultDyes', 'customDyes', 'lockedDyes'];
    for (var i=0; i<dyeTypeOrder.length; i++) {
        var dyeType = dyeTypeOrder[i];
        var dyes = [];
        switch(dyeType) {
            case 'defaultDyes':
                dyes = gearDyeGroups[dyeType];
                break;
            case 'customDyes':
                dyes = shaderDyeGroups[dyeType];
                break;
            case 'lockedDyes':
                dyes = gearDyeGroups[dyeType];
                break;
        }
        for (var j=0; j<dyes.length; j++) {
            var dye = dyes[j];
            resolvedDyes[j] = dye;
        }
    }

    return resolvedDyes;
}
function getGearDyes(gear) {
    var dyeGroups = {
        customDyes: gear.custom_dyes || [],
        defaultDyes: gear.default_dyes || [],
        lockedDyes: gear.locked_dyes || []
    };

    var gearDyeGroups = {};

    for (var dyeType in dyeGroups) {
        var dyes = dyeGroups[dyeType];
        var gearDyes = [];
        for (var i=0; i<dyes.length; i++) {
            var dye = dyes[i];
            var dyeTextures = dye.textures;
            var materialProperties = dye.material_properties;

            var gearDyeTextures = {};

            // Locate referenced textures
            for (var dyeTextureId in dyeTextures) {
                var dyeTexture = dyeTextures[dyeTextureId];

                if (dyeTexture.reference_id && contentLoaded.textures[dyeTexture.reference_id] !== undefined) {
                    gearDyeTextures[dyeTextureId] = contentLoaded.textures[dyeTexture.reference_id];
                }
                else if (dyeTexture.name && contentLoaded.textures[dyeTexture.name] !== undefined) {
                    gearDyeTextures[dyeTextureId] = contentLoaded.textures[dyeTexture.name];
                }
            }

            // Spasm.GearDye
            var gearDye = {
                hash: dye.hash,
                investmentHash: dye.investment_hash,
                slotTypeIndex: dye.slot_type_index,
                variant: dye.variant,

                diffuse: gearDyeTextures.diffuse ? gearDyeTextures.diffuse.texture : null,
                normal: gearDyeTextures.normal ? gearDyeTextures.normal.texture : null,
                decal: gearDyeTextures.decal ? gearDyeTextures.decal.texture : null,

                // Not used?
                primaryDiffuse: gearDyeTextures.primary_diffuse ? gearDyeTextures.primary_diffuse.texture : null,
                secondaryDiffuse: gearDyeTextures.secondary_diffuse ? gearDyeTextures.secondary_diffuse.texture : null,

                isCloth: dye.cloth
            };

            var dyeMat = dye.material_properties;

            switch(game) {
                case 'destiny':
                    gearDye.dyeVariant = dye.variant;
                    gearDye.dyeBlendMode = dye.blend_mode;

                    gearDye.primaryColor = dyeMat.primary_color;
                    gearDye.secondaryColor = dyeMat.secondary_color;

                    gearDye.decalAlphaMapTransform = dyeMat.decal_alpha_map_transform;
                    gearDye.decalBlendOption = dyeMat.decal_blend_option;

                    gearDye.detailNormalContributionStrength = dyeMat.detail_normal_contribution_strength;
                    gearDye.detailTransform = dyeMat.detail_transform;
                    gearDye.specularProperties = dyeMat.specular_properties;
                    gearDye.subsurfaceScatteringStrength = dyeMat.subsurface_scattering_strength;

                    break;
                case 'destiny2':
                    gearDye.primaryColor = dyeMat.primary_albedo_tint;
                    gearDye.secondaryColor = dyeMat.secondary_albedo_tint;
                    gearDye.wornColor = dyeMat.worn_albedo_tint;

                    var spec = dye.material_properties.specular_properties;

                    var emissive = dye.material_properties.emissive_tint_color_and_intensity_bias;

                    gearDye.detailDiffuseTransform = dye.material_properties.detail_diffuse_transform;
                    gearDye.detailNormalTransform = dye.material_properties.detail_normal_transform;

                    // Physically Based Rendering
                    // emissive_pbr_params
                    // lobe_pbr_params
                    // tint_pbr_params
                    break;
            }

            gearDyes.push(gearDye);
        }
        gearDyeGroups[dyeType] = gearDyes;
    }
    return gearDyeGroups;
}

The material_properties differ between Destiny 1 and Destiny 2 as the render in the latter was changed to use Physically Based Rendering. I unfortunately don't know what all these values are for, which impacts the quality of the renderer. I'm hopeful to eventually get some documentation from the Bungie devs and I created an issue on the github.


Parsing Art Regions

From my debugging, the regionIndex seems to refer to specific regions of geometry making it possible to quickly toggle parts like hiding weapon ammo or huds.

function parseArtRegions(artRegionPatterns, gearDyes) {
    var geometryTextures = parseTextures(artRegionPatterns);

    for (var a=0; a<artRegionPatterns.length; a++) {
        var artRegionPattern = artRegionPatterns[a];

        var skipRegion = false;
        switch(artRegionPattern.regionIndex) {
            case -1: // armor (no region)
            case 0: // weapon grip
            case 1: // weapon body
            //case 2: // ??
            case 3: // weapon scope
            case 4: // weapon stock/scope?
            case 5: // weapon magazine
            case 6: // weapon ammo (machine guns)

            case 8: // ship helm
            case 9: // ship guns
            case 10: // ship casing
            case 11: // ship engine
            case 12: // ship body
                break;

            case 21: // hud
                skipRegion = true;
                break;

            case 22: // sparrow wings
            case 23: // sparrow body

            case 24: // ghost shell casing
            case 25: // ghost shell body
            case 26: // ghost shell cube?
                break;
            default:
                console.warn('UnknownArtRegion['+a+']', artRegionPattern.regionIndex);
                break;
        }
        if (skipRegion) continue;

        for (var g=0; g<artRegionPattern.geometry.length; g++) {
            var geometryHash = artRegionPattern.geometry[g];
            var tgxBin = contentLoaded.geometry[geometryHash];

            if (tgxBin == undefined) {
                console.warn('MissingGeometry['+g+']', geometryHash);
                continue;
            }

            parseGeometry(geometryHash, geometryTextures, gearDyes);
        }
    }
}

Parsing Textures

We need to build the texture plates which are made up of sub textures. This code uses a HTML5 Canvas to stitch images together.

function parseTextures(artRegionPatterns) {
    var canvasPlates = {};
    var geometryTextures = [];

    for (var a=0; a<artRegionPatterns.length; a++) {
        var artRegionPattern = artRegionPatterns[a];

        for (var g=0; g<artRegionPattern.geometry.length; g++) {
            var geometryHash = artRegionPattern.geometry[g];

            var tgxBin = contentLoaded.geometry[geometryHash];

            if (!tgxBin) {
                console.warn('MissingTGXBinGeometry['+g+']', geometryHash);
                continue;
            }

            var metadata = tgxBin.metadata;
            var texturePlates = metadata.texture_plates;

            // Spasm.TGXAssetLoader.prototype.getGearRenderableModel
            if (texturePlates.length == 1) {
                var texturePlate = texturePlates[0];
                var texturePlateSet = texturePlate.plate_set;

                // Stitch together plate sets
                // Web versions are pre-stitched
                for (var texturePlateId in texturePlateSet) {
                    var texturePlate = texturePlateSet[texturePlateId];
                    var texturePlateRef = texturePlateId+'_'+texturePlate.plate_index;

                    var textureId = texturePlateId;
                    switch(texturePlateId) {
                        case 'diffuse': textureId = 'map'; break;
                        case 'normal': textureId = 'normalMap'; break;
                        case 'gearstack': textureId = 'gearstackMap'; break;
                        default:
                            console.warn('UnknownTexturePlateId', texturePlateId, texturePlateSet);
                            break;
                    }

                    // Web version uses pre-plated textures
                    var platedTexture = contentLoaded.platedTextures[texturePlate.reference_id];
                    var scale = 1;

                    if (platedTexture) {
                        scale = platedTexture.texture.image.width/texturePlate.plate_size[0];
                    }

                    if (texturePlate.texture_placements.length == 0) {
                        //console.warn('SkippedEmptyTexturePlate['+texturePlateId+'_'+texturePlate.plate_index+']');
                        //continue;
                    }

                    var canvasPlate = canvasPlates[texturePlateRef];
                    if (!canvasPlate) {
                        //console.log('NewTexturePlacementCanvas['+texturePlateRef+']');
                        canvas = document.createElement('canvas');
                        canvas.width = texturePlate.plate_size[0];
                        canvas.height = texturePlate.plate_size[1];
                        ctx = canvas.getContext('2d');

                        ctx.fillStyle = '#000000';
                        ctx.fillRect(0, 0, canvas.width, canvas.height);

                        ctx.fillStyle = '#FFFFFF';
                        canvasPlate = {
                            plateId: texturePlateId,
                            textureId: textureId,
                            canvas: canvas,
                            hashes: []
                        };
                        canvasPlates[texturePlateRef] = canvasPlate;
                    }
                    canvas = canvasPlate.canvas;
                    ctx = canvas.getContext('2d');
                    if (canvasPlate.hashes.indexOf(geometryHash) == -1) canvasPlate.hashes.push(geometryHash);

                    for (var p=0; p<texturePlate.texture_placements.length; p++) {
                        var placement = texturePlate.texture_placements[p];
                        var placementTexture = contentLoaded.textures[placement.texture_tag_name];
                        //VertexColorsent);

                        // Fill draw area with white in case there are textures with an alpha channel
                        //ctx.fillRect(placement.position_x*scale, placement.position_y*scale, placement.texture_size_x*scale, placement.texture_size_y*scale);
                        // Actually it looks like the alpha channel is being used for masking
                        ctx.clearRect(
                            placement.position_x*scale, placement.position_y*scale,
                            placement.texture_size_x*scale, placement.texture_size_y*scale
                        );

                        if (platedTexture) {
                            ctx.drawImage(
                                platedTexture.texture.image,
                                placement.position_x*scale, placement.position_y*scale,
                                placement.texture_size_x*scale, placement.texture_size_y*scale,
                                placement.position_x*scale, placement.position_y*scale,
                                placement.texture_size_x*scale, placement.texture_size_y*scale
                            );
                        } else {
                            // Should be fixed, but add these checks in case
                            if (!placementTexture) {
                                console.warn('MissingPlacementTexture', placement.texture_tag_name, contentLoaded.textures);
                                continue;
                            }
                            if (!placementTexture.texture.image) {
                                console.warn('TextureNotLoaded', placementTexture);
                                continue;
                            }
                            ctx.drawImage(
                                placementTexture.texture.image,
                                placement.position_x, placement.position_y,
                                placement.texture_size_x, placement.texture_size_y);
                        }
                    }
                }
            }
            else if (texturePlates.length > 1) {
                console.warn('MultipleTexturePlates?', texturePlates);
            }
        }
    }

    // Convert canvasPlates to textures
    for (var canvasPlateId in canvasPlates) {
        var canvasPlate = canvasPlates[canvasPlateId];

        // Convert canvas plate to a texture
        var dataUrl = canvasPlate.canvas.toDataURL('image/png');
        loadDataTexture(dataUrl, canvasPlateId, null, true);

        for (var i=0; i<canvasPlate.hashes.length; i++) {
            var geometryHash = canvasPlate.hashes[i];
            if (geometryTextures[geometryHash] == undefined) {
                geometryTextures[geometryHash] = {};
            }

            if (geometryTextures[geometryHash][canvasPlate.textureId] != undefined) {
                console.warn('OverridingTexturePlate['+geometryHash+':'+canvasPlate.textureId+']', geometryTextures[geometryHash][canvasPlate.textureId]);
                continue;
            }

            var texture = contentLoaded.platedTextures[canvasPlateId].texture;
            geometryTextures[geometryHash][canvasPlate.textureId] = texture;
        }
    }

    return geometryTextures;
}

Parsing Geometry

The last step is parsing the geometry data and preparing it for sending to the renderer.

function parseGeometry(geometryHash, geometryTextures, gearDyes) {
    var tgxBin = contentLoaded.geometry[geometryHash];
    var renderMeshes = parseTGXAsset(tgxBin, geometryHash);

    // Prepare all the data for sending to the renderer
}

function parseTGXAsset(tgxBin, geometryHash) {
    var metadata = tgxBin.metadata; // Arrangement

    var meshes = [];

    for (var r=0; r<metadata.render_model.render_meshes.length; r++) {
        var renderMeshIndex = r;
        var renderMesh = metadata.render_model.render_meshes[renderMeshIndex]; // BoB Bunch of Bits

        // IndexBuffer
        var indexBufferInfo = renderMesh.index_buffer;
        var indexBufferData = tgxBin.files[tgxBin.lookup.indexOf(indexBufferInfo.file_name)].data;

        var indexBuffer = [];
        for (var j=0; j<indexBufferInfo.byte_size; j+=indexBufferInfo.value_byte_size) {
            var indexValue = utils.ushort(indexBufferData, j);
            indexBuffer.push(indexValue);
        }

        // VertexBuffer
        var vertexBuffer = parseVertexBuffers(tgxBin, renderMesh);

        var parts = [];
        var partIndexList = [];
        var stagesToRender = [0, 7, 15]; // Hardcoded?
        var partOffsets = [];

        var partLimit = renderMesh.stage_part_offsets[4];//renderMesh.stage_part_list.length;
        for (var i=0; i<partLimit; i++) {
            partOffsets.push(i);
        }

        for (var i=0; i<partOffsets.length; i++) {
            var partOffset = partOffsets[i];
            var stagePart = renderMesh.stage_part_list[partOffset];
            if (!stagePart) {
                console.warn('MissingStagePart['+renderMeshIndex+':'+partOffset+']');
                continue;
            }
            if (partIndexList.indexOf(stagePart.start_index) != -1) {
                //console.warn('DuplicatePart['+renderMeshIndex+':'+partOffset, stagePart);
                continue;
            }
            partIndexList.push(stagePart.start_index);
            parts.push(parseStagePart(stagePart));
        }

        // Spasm.RenderMesh
        meshes.push({
            positionOffset: renderMesh.position_offset,
            positionScale: renderMesh.position_scale,
            texcoordOffset: renderMesh.texcoord_offset,
            texcoordScale: renderMesh.texcoord_scale,
            texcoord0ScaleOffset: renderMesh.texcoord0_scale_offset,
            indexBuffer: indexBuffer,
            vertexBuffer: vertexBuffer,
            parts: parts
        });
    }

    return meshes;
}
// Spasm.RenderMesh.prototype.getAttributes
function parseVertexBuffers(tgxBin, renderMesh) {
    var stagePartVertexStreamLayoutDefinition = renderMesh.stage_part_vertex_stream_layout_definitions[0];
    var formats = stagePartVertexStreamLayoutDefinition.formats;

    var vertexBuffer = [];

    for (var vertexBufferIndex in renderMesh.vertex_buffers) {
        //for (var j=0; renderMesh.vertex_buffers.length; j++) {
        var vertexBufferInfo = renderMesh.vertex_buffers[vertexBufferIndex];
        var vertexBufferData = tgxBin.files[tgxBin.lookup.indexOf(vertexBufferInfo.file_name)].data;
        var format = formats[vertexBufferIndex];

        var vertexIndex = 0;
        for (var v=0; v<vertexBufferInfo.byte_size; v+= vertexBufferInfo.stride_byte_size) {
            var vertexOffset = v;
            if (vertexBuffer.length <= vertexIndex) vertexBuffer[vertexIndex] = {};
            for (var e=0; e<format.elements.length; e++) {
                var element = format.elements[e];
                var values = [];

                var elementType = element.type.replace('_vertex_format_attribute_', '');
                var types = ["ubyte", "byte", "ushort", "short", "uint", "int", "float"];
                for (var typeIndex in types) {
                    var type = types[typeIndex];
                    if (elementType.indexOf(type) === 0) {
                        var count = parseInt(elementType.replace(type, ''));
                        var j, value;
                        switch(type) {
                            case 'ubyte':
                                for (j=0; j<count; j++) {
                                    value = utils.ubyte(vertexBufferData, vertexOffset);
                                    if (element.normalized) value = utils.unormalize(value, 8);
                                    values.push(value);
                                    vertexOffset++;
                                }
                                break;
                            case 'byte':
                                for (j=0; j<count; j++) {
                                    value = utils.byte(vertexBufferData, vertexOffset);
                                    if (element.normalized) value = utils.normalize(value, 8);
                                    values.push(value);
                                    vertexOffset++;
                                }
                                break;
                            case 'ushort':
                                for(j=0; j<count; j++) {
                                    value = utils.ushort(vertexBufferData, vertexOffset);
                                    if (element.normalized) value = utils.unormalize(value, 16);
                                    values.push(value);
                                    vertexOffset += 2;
                                }
                                break;
                            case 'short':
                                for(j=0; j<count; j++) {
                                    value = utils.short(vertexBufferData, vertexOffset);
                                    if (element.normalized) value = utils.normalize(value, 16);
                                    values.push(value);
                                    vertexOffset += 2;
                                }
                                break;
                            case 'uint':
                                for(j=0; j<count; j++) {
                                    value = utils.uint(vertexBufferData, vertexOffset);
                                    if (element.normalized) value = utils.unormalize(value, 32);
                                    values.push(value);
                                    vertexOffset += 4;
                                }
                                break;
                            case 'int':
                                for(j=0; j<count; j++) {
                                    value = utils.int(vertexBufferData, vertexOffset);
                                    if (element.normalized) value = utils.normalize(value, 32);
                                    values.push(value);
                                    vertexOffset += 4;
                                }
                                break;
                            case 'float':
                                // Turns out all that icky binary2float conversion stuff can be done with a typed array, who knew?
                                values = new Float32Array(vertexBufferData.buffer, vertexOffset, count);
                                vertexOffset += count*4;
                                break;
                        }
                        break;
                    }
                }

                var semantic = element.semantic.replace('_tfx_vb_semantic_', '');
                switch(semantic) {
                    case 'position':
                    case 'normal':
                    case 'tangent': // Not used
                    case 'texcoord':
                    case 'blendweight': // Bone weights 0-1
                    case 'blendindices': // Bone indices, 255=none, index starts at 1?
                    case 'color':
                        break;
                    default:
                        console.warn('Unknown Vertex Semantic', semantic, element.semantic_index, values);
                        break;
                }
                vertexBuffer[vertexIndex][semantic+element.semantic_index] = values;
            }
            vertexIndex++;
        }
    }
    return vertexBuffer;
}
// Spasm.RenderablePart
function parseStagePart(stagePart) {
    var gearDyeSlot = 0;
    var usePrimaryColor = true;
    var useInvestmentDecal = false;

    switch(stagePart.gear_dye_change_color_index) {
        case 0:
            gearDyeSlot = 0;
            break;
        case 1:
            gearDyeSlot = 0;
            usePrimaryColor = false;
            break;
        case 2:
            gearDyeSlot = 1;
            break;
        case 3:
            gearDyeSlot = 1;
            usePrimaryColor = false;
            break;
        case 4:
            gearDyeSlot = 2;
            break;
        case 5:
            gearDyeSlot = 2;
            usePrimaryColor = false;
            break;
        case 6:
            gearDyeSlot = 3;
            useInvestmentDecal = true;
            break;
        case 7:
            gearDyeSlot = 3;
            useInvestmentDecal = true;
            break;
        default:
            console.warn('UnknownDyeChangeColorIndex['+stagePart.gear_dye_change_color_index+']', stagePart);
            break;
    }

    var part = {
        externalIdentifier: stagePart.external_identifier,
        changeColorIndex: stagePart.gear_dye_change_color_index,
        primitiveType: stagePart.primitive_type,
        lodCategory: stagePart.lod_category,
        gearDyeSlot: gearDyeSlot,
        usePrimaryColor: usePrimaryColor,
        useInvestmentDecal: useInvestmentDecal,
        indexMin: stagePart.index_min,
        indexMax: stagePart.index_max,
        indexStart: stagePart.start_index,
        indexCount: stagePart.index_count
    };

    return part;
}

Writing a Three.js Plugin

If you've made it to this point, you will know there is a lot involved with loading and parsing gear asset data. In order to make it possible for people who aren't familiar with stuff like 3d rendering or manipulating binary files, we need to create a plugin that handles all of this and is simple to interact with.

var itemHash = 1274330687; // Gjallarhorn
THREE.TGXLoader.APIKey = '{insert-api-key}'; // https://www.bungie.net/en/Application
var loader = new THREE.TGXLoader();
loader.load(itemHash, function(geometry, materials) {
    console.log('LoadedItem', geometry, materials);
    mesh = new THREE.Mesh(geometry, new THREE.MultiMaterial(materials));
    mesh.rotation.x = 90 * Math.PI / 180;
    mesh.scale.set(500, 500, 500);
    scene.add(mesh);
});

That's like the bare minimum you need to load and add Destiny model to a Three.js scene with my TGXLoader plugin. You can find more examples and options in the readme.


Transforming the Data

The first step is to simplify the complexity of the data that will be fed into Three.js. To do this, I had to flatten the entire model into a single THREE.Geometry.

function parseGeometry(geometryHash, geometryTextures, gearDyes) {
    var tgxBin = contentLoaded.geometry[geometryHash];
    var renderMeshes = parseTGXAsset(tgxBin, geometryHash);

    var gearDyeSlotOffsets = [];

    if (loadTextures) {
        for (var i=0; i<gearDyes.length; i++) {
            var gearDye = gearDyes[i];

            gearDyeSlotOffsets.push(materials.length);

            // Create a material for both primary and secondary color variants
            for (var j=0; j<2; j++) {
                var materialParams = {
                    game: game,
                    //side: THREE.DoubleSide,
                    //overdraw: true,
                    skinning: hasBones,
                    //color: 0x777777,
                    //emissive: 0x444444,
                    usePrimaryColor: j == 0,
                    envMap: null
                };
                for (var textureId in geometryTextures[geometryHash]) {
                    var texture = geometryTextures[geometryHash][textureId];

                    materialParams[textureId] = texture;
                }

                copyGearDyeParams(gearDye, materialParams);

                var material = new THREE.TGXMaterial(materialParams);
                material.name = geometryHash+'-'+(j == 0 ? 'Primary' : 'Secondary')+i;
                materials.push(material);
            }
        }
    }

    for (var m=0; m<renderMeshes.length; m++) {
        var renderMesh = renderMeshes[m];
        var indexBuffer = renderMesh.indexBuffer;
        var vertexBuffer = renderMesh.vertexBuffer;
        var positionOffset = renderMesh.positionOffset;
        var positionScale = renderMesh.positionScale;
        var texcoordOffset = renderMesh.texcoordOffset;
        var texcoordScale = renderMesh.texcoordScale;
        var parts = renderMesh.parts;

        if (parts.length == 0) {
            console.log('Skipped RenderMesh['+geometryHash+':'+m+']: No parts');
            continue;
        } // Skip meshes with no parts

        // Spasm.Renderable.prototype.render
        var partCount = -1;
        for (var p=0; p<parts.length; p++) {
            var part = parts[p];

            if (!checkRenderPart(part)) continue;
            partCount++;

            var gearDyeSlot = part.gearDyeSlot;

            if (gearDyeSlotOffsets[gearDyeSlot] == undefined) {
                console.warn('MissingDefaultDyeSlot', gearDyeSlot);
                gearDyeSlot = 0;
            }
            var materialIndex = gearDyeSlotOffsets[gearDyeSlot]+(part.usePrimaryColor ? 0 : 1);

            // Load Material
            if (loadTextures) {
                var material = parseMaterial(part, gearDyes[gearDyeSlot], geometryTextures[geometryHash]);

                if (material) {
                    material.name = geometryHash+'-CustomShader'+m+'-'+p;
                    materials.push(material);
                    materialIndex = materials.length-1;
                }
            }

            // Load Vertex Stream
            var increment = 3;
            var start = part.indexStart;
            var count = part.indexCount;

            // PrimitiveType, 3=TRIANGLES, 5=TRIANGLE_STRIP
            // https://stackoverflow.com/questions/3485034/convert-triangle-strips-to-triangles

            if (part.primitiveType === 5) {
                increment = 1;
                count -= 2;
            }

            for (var i=0; i<count; i+= increment) {
                var faceVertexNormals = [];
                var faceVertexUvs = [];
                var faceVertex = [];

                var faceColors = [];

                var detailVertexUvs = [];

                var faceIndex = start+i;

                var tri = part.primitiveType === 3 || i & 1 ? [0, 1, 2] : [2, 1, 0];

                for (var j=0; j<3; j++) {
                    var index = indexBuffer[faceIndex+tri[j]];
                    var vertex = vertexBuffer[index];
                    if (!vertex) { // Verona Mesh
                        console.warn('MissingVertex['+index+']');
                        i=count;
                        break;
                    }
                    var normal = vertex.normal0;
                    var uv = vertex.texcoord0;
                    var color = vertex.color0;

                    var detailUv = vertex.texcoord2;
                    if (!detailUv) detailUv = [0, 0];

                    faceVertex.push(index+vertexOffset);
                    faceVertexNormals.push(new THREE.Vector3(-normal[0], -normal[1], -normal[2]));

                    // TODO There's currently a bug with detail UVs
                    var uvu = uv[0]*texcoordScale[0]+texcoordOffset[0];
                    var uvv = uv[1]*texcoordScale[1]+texcoordOffset[1];
                    faceVertexUvs.push(new THREE.Vector2(uvu, uvv));

                    if (color) {
                        faceColors.push(new THREE.Color(color[0], color[1], color[2]));
                    }

                    detailVertexUvs.push(new THREE.Vector2(
                        uvu*detailUv[0],
                        uvv*detailUv[1]
                    ));
                }
                var face = new THREE.Face3(faceVertex[0], faceVertex[1], faceVertex[2], faceVertexNormals);
                face.materialIndex = materialIndex;
                if (faceColors.length > 0) face.vertexColors = faceColors;
                geometry.faces.push(face);
                geometry.faceVertexUvs[0].push(faceVertexUvs);

                if (geometry.faceVertexUvs.length < 2) geometry.faceVertexUvs.push([]);
                geometry.faceVertexUvs[1].push(detailVertexUvs);
            }
        }

        for (var v=0; v<vertexBuffer.length; v++) {
            var vertex = vertexBuffer[v];
            var position = vertex.position0;
            var x = position[0];
            var y = position[1];
            var z = position[2];
            if (platform == 'web') { // Ignored on mobile?
                x = x*positionScale[0]+positionOffset[0];
                y = y*positionScale[1]+positionOffset[1];
                z = z*positionScale[2]+positionOffset[2];
            }
            geometry.vertices.push(new THREE.Vector3(x, y, z));

            // Set bone weights
            var boneIndex = position[3];
            //var bone = geometry.bones[boneIndex];

            var blendIndices = vertex.blendindices0 ? vertex.blendindices0 : [boneIndex, 255, 255, 255];
            var blendWeights = vertex.blendweight0 ? vertex.blendweight0: [1, 0, 0, 0];

            var skinIndex = [0, 0, 0, 0];
            var skinWeight = [0, 0, 0, 0];

            var totalWeights = 0;
            for (var w=0; w<blendIndices.length; w++) {
                if (blendIndices[w] == 255) break;
                skinIndex[w] = blendIndices[w];
                skinWeight[w] = blendWeights[w];
                totalWeights += blendWeights[w]*255;
            }
            if (totalWeights != 255) console.error('MissingBoneWeight', 255-totalWeights, i, j);

            geometry.skinIndices.push(new THREE.Vector4().fromArray(skinIndex));
            geometry.skinWeights.push(new THREE.Vector4().fromArray(skinWeight));
        }
        vertexOffset += vertexBuffer.length;
    }
}
The detail UV issue causes some textures to scale improperly.

Triangle strip primitive types are not supported in Three.js, so I had to convert them to triangle primitives. Don't ask me to explain how the code works, I just grabbed it off of Stack Overflow.

The concept of gear dyes doesn't exist in Three.js so they need to be converted to materials, which also means I needed to write some custom shaders to handle the data. I still need to work on the code for this, which has been halted due to not having any documentation on all the material properties I mentioned earlier.

Because I am flattening everything into a single geometry, I have to create a separate material for every variation used.

var gearDyeSlotOffsets = [];

if (loadTextures) {
    for (var i=0; i<gearDyes.length; i++) {
        var gearDye = gearDyes[i];

        gearDyeSlotOffsets.push(materials.length);

        // Create a material for both primary and secondary color variants
        for (var j=0; j<2; j++) {
            var materialParams = {
                game: game,
                //side: THREE.DoubleSide,
                //overdraw: true,
                skinning: hasBones,
                //color: 0x777777,
                //emissive: 0x444444,
                usePrimaryColor: j == 0,
                envMap: null
            };
            for (var textureId in geometryTextures[geometryHash]) {
                var texture = geometryTextures[geometryHash][textureId];

                materialParams[textureId] = texture;
            }

            copyGearDyeParams(gearDye, materialParams);

            var material = new THREE.TGXMaterial(materialParams);
            material.name = geometryHash+'-'+(j == 0 ? 'Primary' : 'Secondary')+i;
            materials.push(material);
        }
    }
}

Parsing Materials

There are multiple different types of shaders used to render gear in Destiny. While I have figured out how to implement some of them, the issue lies with not knowing how to interpret the data to determine which shader to use.

function parseMaterial(part, gearDye, textures) {
    // TODO Figure out how to properly determine what shader to use!
}
The Gearstack: Uses a channel mask to apply color and detail to the material. The ghost shell iris is a custom shader. The Vex Mythoclast has lots of glass materials that emit an orange glow. I'm not certain how the color is being applied yet either. The Sunbreakers use multiple shaders allow Warlocks to harness the power of the Sun. When a Gunslinger's super is fully charged, the Celestial Nighthawk's eagle eyes glow. Destiny 2 models use Physically Based Rendering to more accurately simulate lighting. The mathematically correct color for gold is yellow vomit.

Writing Custom Shaders

Three.js allows you to create custom shaders through THREE.ShaderMaterial. However by default it's the equivalent of hitting "New" and starting with a blank document. I'm still learning about shaders and I'm in no rush to write one completely from scratch, so I chose to extend an existing one and add custom code to it. The way this is done currently is...a bit messy. In order to add the necessary changes to the built in shaders, I needed to figure out how to inject it into the raw shader code.

TGXMaterial.prototype.insertBefore = function(search, shader, insertCode) {
    search += "\n";
    if (typeof insertCode != 'string') insertCode = insertCode.join("\n")+"\n";
    shader = shader.replace(search, insertCode+search);
    return shader;
};
TGXMaterial.prototype.insertAfter = function(search, shader, insertCode) {
    search += "\n";
    if (typeof insertCode != 'string') insertCode = insertCode.join("\n")+"\n";
    shader = shader.replace(search, search+insertCode);
    return shader;
};
TGXMaterial.prototype.replace = function(search, shader, insertCode) {
    search += "\n";
    if (typeof insertCode != 'string') insertCode = insertCode.join("\n")+"\n";
    shader = shader.replace(search, insertCode);
    return shader;
};

Fragment shader code for setting up parameters as well as some simple blending functions most of which are based on Photoshop layer effects.

var gearstackParsFragment = [
    //"#define saturate(value) clamp(value, 0.0, 1.0)",
    "const float gamma_correction_power = 2.2;",
    "const float gamma_correction_power_inverse = 1.0/2.2;",

    // Blend Functions
    "vec4 blend_overlay(vec4 back, vec4 front) {",
        "return front * saturate(back * 4.0) + saturate(back - 0.25);",
    "}",
    "vec4 blend_multiply(vec4 back, vec4 front) {",
        "return back * front;",
    "}",
    "vec4 blend_screen(vec4 back, vec4 front) {",
        "vec4 back_screen = vec4(1.0 - back.x, 1.0 - back.y, 1.0 - back.z, 1.0);",
        "vec4 front_screen = vec4(1.0 - front.x, 1.0 - front.y, 1.0 - front.z, 1.0);",
        "vec4 screen = back_screen * front_screen;",
        "return vec4(1.0 - screen.x, 1.0 - screen.y, 1.0 - screen.z, 1.0);",
    "}",
    "vec4 blend_hard_light(vec4 back, vec4 front) {",
        "return vec4(",
            "front.x < 0.5 ? (2.0 * back.x * front.x) : (1.0 - 2.0 * (1.0 - back.x) * (1.0 - front.x)),",
            "front.y < 0.5 ? (2.0 * back.y * front.y) : (1.0 - 2.0 * (1.0 - back.y) * (1.0 - front.y)),",
            "front.z < 0.5 ? (2.0 * back.z * front.z) : (1.0 - 2.0 * (1.0 - back.z) * (1.0 - front.z)),",
            "1.0",
        ");",
    "}",

    // Gearstack Fragment Vars
    "#ifdef USE_GEARSTACKMAP",
        "uniform sampler2D gearstackMap;",
    "#endif",

    "#ifdef USE_DESTINY",
        "uniform float blendMode;",
        "uniform bool usePrimaryColor;",
        "uniform vec3 primaryColor;",
        "uniform vec3 secondaryColor;",
    "#endif",

    "#ifdef USE_DESTINY2",
        "uniform bool usePrimaryColor;",
        "uniform vec3 primaryColor;",
        "uniform vec3 secondaryColor;",
        "uniform vec3 wornColor;",
    "#endif",

    // Texture Detail Fragment Vars
    "#ifdef USE_DETAIL",
        "uniform sampler2D detailMap;",
        "uniform sampler2D detailNormalMap;",
        "varying vec2 vUv2;",
    "#endif",

    "#ifdef USE_DECAL",
        "uniform sampler2D detailDecalMap;",
    "#endif",
];
fragmentShader = this.insertAfter('#include <map_pars_fragment>', fragmentShader, gearstackParsFragment);

The other half of the fragment shader that partially implements parts of the gearstack.

var gearstackFragment = [
    "diffuseColor = pow(diffuseColor, vec4(gamma_correction_power));",

    "vec4 gearstackColor = vec4(1.0, 1.0, 1.0, 1.0);",
    "vec4 dyeColor = vec4(1.0, 1.0, 1.0, 1.0);",

    "#ifdef USE_GEARSTACKMAP",
        "gearstackColor = texture2D(gearstackMap, vUv);",
        "dyeColor = usePrimaryColor ? vec4(primaryColor, 1.0) : vec4(secondaryColor, 1.0);",
    "#endif",

    // Dye Textures (Detail)
    "#ifdef USE_DETAIL",
        "vec4 color_dye_diffuse_texture = texture2D(detailMap, vUv2);",
        //"vec4 color_dye_diffuse_texture = texture2D(u_texture_dye_diffuse, v_texcoord2);",

        "float dye_alpha = color_dye_diffuse_texture.w;",
        "float dye_color_normalize = (1.0 - dye_alpha) * 0.5;",
        "vec4 color_dye_diffuse = pow(vec4("
            +"color_dye_diffuse_texture.x * dye_alpha + dye_color_normalize, "
            +"color_dye_diffuse_texture.y * dye_alpha + dye_color_normalize, "
            +"color_dye_diffuse_texture.z * dye_alpha + dye_color_normalize, 1.0), "
            +"vec4(gamma_correction_power));",
        //"diffuseColor = blend_overlay(color_dye_diffuse, diffuseColor);",

        // TODO figure out how to make decals look worn
        "#ifdef USE_DECAL",
            "vec4 decalColor = texture2D(detailDecalMap, vUv2);",
            //"diffuseColor = blend_multiply(decalColor, diffuseColor);",
        "#endif",

        //"vec4 color_dye_normal = texture2D(dyeNormal, vUv2);",
        //"color_dye_normal = color_dye_normal * 2.0 - 1.0;",
        //"normal = normal + color_dye_normal.xy;",
    "#endif",

    "#ifdef USE_DESTINY",
        "vec4 blendColorUncorrected = mix(diffuseColor, blend_overlay(diffuseColor, dyeColor), gearstackColor.r);",
        "diffuseColor = blendColorUncorrected;",

        // Worn Color
        //"vec4 detailColor = vec4(wornColor, 1.0);",
        //"vec4 blendDetail = mix(diffuseColor, blend_overlay(diffuseColor, detailColor), dyeAmbientColor.b);",
        //"diffuseColor = blendDetail;",

        //"vec4 decalColor = vec4(1.0, 0.0, 1.0, 1.0);",
        //"vec4 blendDecal = mix(diffuseColor, blend_multiply(diffuseColor, decalColor), gearstackColor.b);",
        //"diffuseColor = blendDecal;",
    "#endif",

    "#ifdef USE_DESTINY2",
        // Gearstack Textures
        // Notes from https://twitter.com/HashtagVeegie/status/929245226207649792
        // Red is AO, Green is smoothness, Blue is encoded alpha test and emissive.
        // Alpha is encoded dye mask, non-dyed metalness, and wear mask.

        "vec4 blendColorUncorrected = mix(diffuseColor, blend_overlay(diffuseColor, dyeColor), gearstackColor.r);",
        "diffuseColor = blendColorUncorrected;",
    "#endif",

    "diffuseColor = vec4(pow(diffuseColor.xyz, vec3(gamma_correction_power_inverse)), 1.0);",

    "#ifdef USE_ALPHATESTSTACK",
        "#ifdef USE_DESTINY",
            //"diffuseColor.a = gearstackColor.b;",
            "diffuseColor.a = diffuseColor.g;",
            "if (diffuseColor.a < 1.0 - gearstackColor.b) discard;",
        "#endif",
    "#endif",
];
fragmentShader = this.insertAfter('#include <map_fragment>', fragmentShader, gearstackFragment);

I've also been working on applying other material properties to improve the rendering however without knowing what all the values in the data mean, it's guess work.

var specularFragment = [
    "#ifndef USE_SPECULARMAP",
        "specularStrength = 1.0;",
        "#ifdef USE_DESTINY",
            "specularStrength = gearstackColor.g * 0.2;",
        "#endif",
        //"#ifdef NO_SHINE",
        //  "specularStrength = 0.0;",
        //"#endif",
    "#endif"
];

fragmentShader = this.insertAfter('#include <specularmap_fragment>', fragmentShader, specularFragment);

Hopefully once I can get a hold of some documentation, I can implement a more complete shader material. I have some understanding of Physically Based Rendering however I'm not sure which values define stuff like how metallic or shiny an object is. I could continue to work at improving some aspects of the rendering, but in doing so I will likely break other things.


Adding Support For Loadouts

Up until now, we have only been looking at loading one piece of gear at a time, but it'd be even more useful to be able to load an entire loadout all at once!

var options = {
    itemHashes: [
        838933125, // Celestial Nighthawk
        2217280775, // Sealed Ahamkara Grasps
        2882684152, // Crest of Alpha Lupi
        1775312683, // Bones of Eao
        2300914893, // Cloak of Oblivion
    ],
    shaderHash: 3367786034 // Goldspiral Shader
};

THREE.TGXLoader.APIKey = '{insert-api-key}'; // https://www.bungie.net/en/Application
var loader = new THREE.TGXLoader();
loader.load(options, function(geometry, materials) {
    console.log('LoadedLoadout', geometry, materials);
    mesh = new THREE.Mesh(geometry, new THREE.MultiMaterial(materials));
    mesh.rotation.x = 90 * Math.PI / 180;
    mesh.scale.set(500, 500, 500);
    scene.add(mesh);
});

Even though we are now loading multiple items at once, the Three.js interface remains largely the same!

Hunter class wearing a full set of exotic gear.

Bonus: Animating Characters

The spasm library actually implemented character animations on the old profile page and later for previewing Eververse taunts on the website. The animation data isn't as readily available though because it was manually built and required extra work to add them to the manifest. There's a certain property you can check on DestinyInventoryItemDefinitions to see if a taunt has an animation available, and there's also a list of taunts available for Destiny 1.

Unfortunately there's a bit more work involved with converting the animation data to something Three.js understands!


Bonus #2: Web VR

Fellow community developer Chris Fried has put together a project that leverages on the TGXLoader to put Destiny models into Web VR!


Final Thoughts

This has been an interesting and fun project to work on that has helped better my understanding of 3d rendering. It is also interesting to see how a AAA company stores and manages the 3d data for the web. I definitely plan on revisiting this article as more information is made available over time. I'm also very eager to see how other developers utilize my Three.js plugin in their own projects, Chris Fried in particular has come up with some of the coolest ideas!

If you happen to know something I don't, or would like to contribute or get in contact, feel free to send me a message or check out the source code on the github!


References

  1. http://advances.realtimerendering.com/destiny/gdc_2017/
  2. http://advances.realtimerendering.com/destiny/siggraph2014/
  3. http://www.gdcvault.com/play/1020412/Building-Customizable-Characters-for-Bungie
  4. https://destinydevs.github.io/BungieNetPlatform/docs/Getting-Started
  5. https://destinydevs.github.io/BungieNetPlatform/docs/Manifest
  6. https://destinydevs.github.io/BungieNetPlatform/docs/services/Destiny/Destiny-GetDestinySingleDefinition
  7. https://en.wikipedia.org/wiki/Normal_number_(computing)
  8. https://github.com/Bungie-net/api/issues/422
  9. https://github.com/Bungie-net/api/wiki/Bungie.net-Application-Portal
  10. https://github.com/lowlines/three-tgx-loader
  11. https://github.com/lowlines/three-tgx-loader/blob/master/gistfile1.txt
  12. https://github.com/lowlines/three-tgx-loader/tree/master/bnet-src
  13. https://lowlidev.com.au/destiny/gear-viewer
  14. https://stackoverflow.com/questions/3485034/convert-triangle-strips-to-triangles
  15. https://threejs.org/
  16. https://threejs.org/docs/#api/core/Geometry
  17. https://threejs.org/docs/#api/materials/ShaderMaterial
  18. https://twitter.com/EdgarVerona
  19. https://twitter.com/HashtagVeegie
  20. https://twitter.com/HashtagVeegie/status/929245226207649792
  21. https://twitter.com/chrisfried
  22. https://twitter.com/guardtheater/status/925843775116718080
  23. https://twitter.com/guardtheater/status/927636336743534597
  24. https://twitter.com/guardtheater/status/927706230658871301
  25. https://twitter.com/guardtheater/status/928292028370358272
  26. https://twitter.com/guardtheater/status/946133638722326528
  27. https://twitter.com/lowlines
  28. https://twitter.com/vivekhnz
  29. https://twitter.com/vivekhnz/status/928826751936905216
  30. https://www.bungie.net/en/Application
  31. https://www.bungie.net/en/Companion
  32. https://www.bungie.net/en/News/Article/13117
  33. https://www.reddit.com/r/DestinyTheGame/comments/3qk93l/bungienets_new_gear_manager_beta/
  34. https://www.youtube.com/watch?v=ryRKv8qBIJw
Like the stuff that I do?  You can now support me through my   :)