3D Model Guide

From XentaxWiki
Jump to: navigation, search

This is a work-in-progress and open to any edits. You may add any information that you feel would be useful, and modify any existing information that may be inaccurate or could be improved. No permission is required. Of course, as this is an open page, "If you don't want your writing to be edited mercilessly, then don't submit it here."

Preface

This page will briefly describe how to reverse a 3D model format. This will not be a tutorial on how to read hex, but instead on various concepts used in 3D computing and rendering that will help make sense of what the data represents.

For a more detailed list of 3D concepts, check out the 3D model glossary

Introduction

Please read the Definitive Guide to Exploring File Formats before starting this guide, which is aimed specifically for 3d file formats. It is assumed that you know how to read files in hex viewers.

No knowledge in 3d is needed, however, any experience in 3d modelling or 3d programming will help. No advance math knowledge is used in 3d formats, though it is in 3d programming, but those are usually hidden inside 3d libraries you use, like your 3d engine or your 3d modeller's scripting language. You should be able to understand what vectors and matrices are though.

Common data

The following a list of things that you should look out for when you're exploring a 3D format. Note that in addition to data related to 3D models, the developers may choose to store other data with it that are only useful for to their engine or scripts. This miscellaneous data typically is not important and can be skipped by your parser.

Each item will also indicate how they are "typically" represented. Note that very often, 8-byte, 4-byte, or 2-byte floats may be used for precision or to optimize space. But 4-byte floats are more often used, so when you're working with float data, if one type doesn't work try another.

Vertices

Data that is usually stored with vertices.

  • Vertex coordinates. Three floats
  • Vertex normals. Three floats
  • Texture coordinates UV(W). Two floats. Three if the W-value is specified.
  • Vertex colors. Four floats
  • Vertex weights
  • Bone indices

Faces

Data that may be stored with faces.

  • Vertex indices
  • Normal indices
  • UV indices
  • Material indices
  • Face Normals (in addition to each vertex having a normal, each face can also have a normal)

Bones

  • Bone index
  • Parent bone index
  • Bone name
  • Bone transformation.

Animation

Morphs

Materials

Exploring a Model Format

Where do I begin?

Naturally, if you have no experience with format reversing, you will probably have no clue where to start. Even if you do, you probably still won't know where to start. Some things that you might want to consider before tackling your own format:

  • Get some decent tools. Some are better than others.
  • Familiarize yourself with the API you're working with
  • Look at existing formats and write a parser for it
  • Get a feel for various patterns that arise in various formats. After awhile these things become second-nature.

Common Patterns

Design patterns often arise in file formats, due to various reasons. This is also true in model formats.

Chunk-based format

This format essentially structures all of the data into data "chunks" or "blocks", where each chunk is identified by an ID and often has a size associated with it.

"In this kind of file structure, each piece of data is embedded in a container that contains a signature identifying the data, as well the length of the data (for binary encoded files). This type of container is called a "chunk". The signature is usually called a chunk id, chunk identifier, or tag identifier." -- Wikipedia

It is usually easy to identify this format because strings are usually used as the chunk ID, but numbers can be used as well.

If you suspect that the format uses a chunk-based format, consider examining a few of the values that you believe are the chunk sizes and see if you can parse the entire file without having to understand any of the data.

Offset tables

Material index ranges

Face material information may be stored in the form of index or face ranges. For example, if a particular mesh is composed of 3000 face indices and used 3 materials, the first material might be assigned to indices 0 to 799, the second material from 800 to 2045, and the third material from 2046 to 2999 (assuming zero-based indexing). If you find yourself with some unknown integers that add up to the total number of indices or faces, you might want to try this out.

Techniques

Some techniques that you might use to help figure out what kind of data you might be looking at.

Using a calculator

Comparing multiple files

A simple model format

Here is a simple model format: http://forum.xentax.com/viewtopic.php?f=29&t=3739 (temporarily just linking to an existing tutorial)

Tools

You will need two things

  • A hex viewer
  • A 3D model viewer/ 3D modeller

This is the bare minimum required to get started on your model reversing journey, but your choice of tools may make your experience easier or more challenging. Of course if you are working with a text format you won't need a hex viewer, but more often than not you will be working with binary formats.

Some common tools are listed below. It is by no means an exhaustive list. Try different tools to get a feel for what you like. Links are available at the end of the page.

Hex Viewers

  • 010 Editor
    • 30-day trial, US$50 license
    • Comes with many useful features.
  • HxD
    • Free
    • Simple hex editor.
  • Hex Workshop
    • 30-day trial, US$90 license

3D Model Viewers/ 3D Modellers

There are many packages for model editing. There are dozens of custom model viewers. The more common ones are as follows:

  • 3D Studio Max
    • 30-day trial, US$3500 license
    • Comes with its own scripting language for importing models.
  • Blender
    • Free
    • Uses Python scripts to import models.
  • Noesis
    • Free
    • Provides C++ and python API's for importing models.
  • Metasequoia
    • Free*, US$45 license
    • Simple 3D editor. Does not support too many things, but it gets the geometry out. The MQO format is very simple as well.
Note that the free version comes with limited features.

Links