Reflections Chunk File

From XentaxWiki
Revision as of 08:09, 14 December 2014 by Red (talk | contribs) (Created page with "==Description== The '''Chunk''' ''(CHNK)'' file format was created by Reflections sometime during 2003. Since then, their games have used thi...")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Description

The Chunk (CHNK) file format was created by Reflections sometime during 2003. Since then, their games have used this "File Chunking" technique to store various types of game data into one file instead of being spread out over multiple files.

Games that are known to use this format include:

  • Driv3r
  • Driver: Parallel Lines
  • Driver 76
  • Driver: San Francisco
  • Driver: Renegade 3D (3ds)
  • Emergency Heroes (Wii)
  • Monster 4x4: Stunt Racer (Wii)

How It Works

A chunk file is also known as a spooler, and contains spoolable resources which can be requested by the game at any time.

"Spooling works like a typical request queue or spool where data, instructions and processes from multiple sources are accumulated for execution later on. Generally, the spool is maintained on the computer’s physical memory, buffers or the I/O device-specific interrupts. The spool is processed in ascending order, working on the basis of a FIFO (first in, first out) algorithm."[1]

This holds true in the sense that a single chunk file may contain various types of data, which can be read from the game at any given time. This allows for faster execution times and requires only one file to be open at a time. Take for example the vehicles in a game - there are many different types of vehicles, which would typically be stored in a folder that contains all necessary files. However, spooling removes the need to have possibly thousands of files in a "vehicles" directory and instead compacts them into a single chunk file. If used correctly, a lot of disk space can be easily saved.

In Driver: Parallel Lines, there is support for compressed chunks. These are used to compress the GPU shaders. Currently, it is unknown what compression method is used.

Specification Breakdown

All data contained within a chunk file is derived from a Spooler. This means a Chunk should inherit from the Spooler type.

Spooler

Offset Type Size Name Description
- Chunk - Parent The parent of the spooler. This value is not represented in the file itself, but is usually assumed by tools while reading children spooler entries. This is important to keep track of, especially for size calculations and retrieving the base offset of a spooler.
0x00 int 4 Magic The identifier for the spooler, which indicates what sort of data this spooler contains. In most cases, this takes the appearance of four uppercase characters, for example "MDPC", used to designate a ModelPackagePC.
0x04 int 4 Offset The byte-aligned offset of the spooler, relative to the absolute offset of its parent chunk. Usually calculated by tools, taking into consideration all children spoolers (if any) preceding the spooler.
0x08 byte 1 Reserved The reserved byte of the spooler. This is paired with the magic number to give this spooler a unique identity. For example, the "MDXN" (RIModelPC) in Driver: Parallel Lines is significantly different than the one in Driver: San Francisco. To differentiate the two, the reserved bytes for these formats are different.
0x09 byte 1 StrLen The length of the description for the spooler. This is used by tools in size calculation operations (creating/modifying data). The description is located directly after the data (Offset + Size).
0x0A byte 1 Alignment The alignment of the spooler. This value represents how many times the number 1 is bit-shifted to the left, for example a value of 12 (0xC) means that the equation (1 << 12) equals an alignment of 4096.

When a chunk needs to calculate its size, each spooler's alignment is taken into consideration. Depending on the position of this spooler, the alignment is used to either calculate the next spooler's offset, or is used to align the chunk's size.

Alignment values known to have been used include:

  • 2 - 4-byte alignment
  • 4 - 16-byte alignment
  • 11 - 2048-byte alignment
  • 12 - 4096-byte alignment
0x0B byte 1 PadByte This padding byte is unused and has no effect on spoolers. Its sole purpose is to put the Size property on an alignment-boundary.

Common padding bytes include 0xCC and 0x00. However, the actual byte used doesn't matter.

0x0C uint 4 Size The computed size of this spooler. It takes into account all of the data contained within the spooler, whether it is just a regular buffer, or a chunk. For chunks, this size is the same as the chunk itself. The description length is not included in this size.

Chunks are the basis of which data is contained. Chunks are a special type of spooler which contain a list of child spoolers.

To understand how this works, compare chunks to a container - it can hold different items (spoolers), or combine multiple containers (chunks) into one. A magic marker identifies the contents, and if needed, a short description of them.

This is how a spooler might look in Pseudo-C:

struct Spooler {
  Chunk parent;
  int offset;
  byte reserved;
  byte strLen;
  byte alignment;
  byte padByte: 0xCC; // put 'size' on an alignment-boundary
  int size;
};

Chunk

Offset Type Size Name Description
0x00 uint 4 Magic The magic number of the chunk. This is always 0x4B4E4843, or CHNK. Tools should check for this magic number before reading data from a spooler, and handle them accordingly.
0x04 int 4 Size The size of the chunk. This is the size of the header, all of the children spoolers (if any), and the alignment padding in-between spoolers. The size of the chunk is padded to the last spooler's alignment.
0x08 int 4 Count The number of children spoolers in the chunk.
0x0C int 4 Reserved The version/type of the chunk. In most cases, this value is 3, except for compressed chunks, which use 0x80000003.
0x10 Spooler[] Count * 16 Children The children spooler entires in the chunk. Each entry corresponds to the Spooler format defined above.

It is important to note that all data within a chunk file is padded to a certain alignment. In the majority of cases, the game will store important information in the padding, and may cause unexpected errors if the padding is eliminated. Tools should take into account the original padding and re-use it for size/offset calculations.

This is how a chunk might look in Pseudo-C:

struct SpoolableChunk : Spooler {
    int magic: 0x4B4E4843; // 'CHNK'
    int size; // file size
    int count; // number of children spoolers
    int reserved: 3; // version/type, depending on usage
    
    Spooler children[];
};