Talk:Guild Wars DAT

From XentaxWiki
Jump to: navigation, search

--Lunarman

I'm just a casual Guild Wars player but I'm interested in modding this game like I've modded many others. On GwG we've found a new program called Texmod which we've been using for ripping texture out the game, I have a feeling it's not taking actual texture files though. [1] is the first page. Sadly we can only modify textures and I'm hoping to move onto models. From what I understand of what you guys here have been saying: you've got a decompressor but no recompressor (so any modifications made can't be seen in game). I've downloaded the decompressor but can't use it (i'm the kinda person that needs a tut).

Anyway, thankyou. I'll be keeping an eye on things here


Preliminary specs

I have added my preliminary specs for this Guild Wars .DAT format. Feel free to update it as necessary, or discuss it here. For starters, the files in there are compressed, but I don't know how. I tried Zlib, to no avail. --Mr.Mouse


--SilentAvenger I disagree with some of the specs there. Using filemon I looked at how GW opens the file. First, I dont know if the header size is really that, as GW just reads the first 32 bytes straight up. Second of all, ffna tables are not neccesserily 512 bytes in size, but rather about 50 bytes, varying a bit. The stuff is stored in 256 blocks in the file, padding the end if needed. Also, there is no ffna offset in the header. Maybe only an offset to the first block. In an empty archive (achieved by killing the internet connection just as GW starts), there is no ffna block, and it still is 00 02 00 01.

Method that GW opens the file, as I have managed to figure out until now: Read first 32 bytes. Go to the MFT pointed by the offset at 0x10 from file start. Read 384 bytes of that MFT, then either (I dont know which yet) - figure out the size by looking at the self reference entry or - Take the int at 0x0c from MFT start, multiply by 24 (the size for a file reference) Take the figure out size, subtract 384, read. (This reads all the MFT) Then, read the block pointed at by the other reference, before the self reference. This block, to my tests is neither compressed data, MFT, ffna, or ATEXDXT* (The four types of data you recognize on sight). Thats as far as I went.

Also: ffna's do not store data. Data seems to be stored in compressed blocks, all ending with 08 00 01 80, followed by what I assume is the uncompressed file size. I have not been able to decompress this, but its probably deflate or decompress. Someone told me by some obscure method of reasoning that it might be the method used by gzip, but he then disconnected from mIRC.

There are ATEXDXT* blocks (* being a digit), which a friend of mine thought are uncompressed DXT* textures.

Some ffna's seem to contain a self-reference entry.

About that last uint32 at each reference. I looked at the dat I had, ran some stat checking (will run on a bigger archive once GW stops downloading), it is unique for every entry *not* pointing to block that starts with ffna, and repeats for the ffna blocks. So, it might not be hash.

Wasnt sure if to post this here or in Talk, tell me for next time :)


--Captain Hey man, if you don't agree with the contents, feel free to add corrections to what the current page is. The common goal is to figure this thing out, and all help is appreciated. :)


--Mr.Mouse I'd rather the discussion takes place here. If you have an alternative spec,please add it below the former one, name it Alternative 1. ;) I'll get back to this discussion soon.

As a matter of fact here. :)

I disagree with some of the specs there. Using filemon I looked at how GW opens the file. First, I dont know if the header size is really that, as GW just reads the first 32 bytes straight up. Second of all, ffna tables are not neccesserily 512 bytes in size, but rather about 50 bytes, varying a bit. The stuff is stored in 256 blocks in the file, padding the end if needed. Also, there is no ffna offset in the header. Maybe only an offset to the first block. In an empty archive (achieved by killing the internet connection just as GW starts), there is no ffna block, and it still is 00 02 00 00.

1. Coders are sometimes weird. They may read the 32 bytes straight up, but they may have editors that don't do that. They may have noticed it never changes and just read it straight up and did not bother to remove the specs code. Also, it's just one file, so we have nothing to compare it with (or do we? Let me know).

2.You have seen ffna tables that are not 512 in size (so from the start to where the actual files begin)? I have just let it download a bit (to about 40 mb) and that is what I noticed. Coincidence that the value I saw was 512? I think not. You see, it resembles perfectly the order the other entries (offset , then size) of other tables. It's intriguing.

3. Yes, the blocks are padded to 256 probably. Should you not wait though until you have the complete .DAT file? Why do you break off the download? Is not the final file the one you play with? The specs you should retrieve from that file I think. Not from an unfinished one. Or am I mistaken?


Method that GW opens the file, as I have managed to figure out until now Read first 32 bytes. Go to the MFT pointed by the offset at 0x10 from file start. Read 384 bytes of that MFT, then either (I dont know which yet) - figure out the size by looking at the self reference entry or - Take the int at 0x0c from MFT start, multiply by 24 (the size for a file reference) Take the figure out size, subtract 384, read. (This reads all the MFT) Then, read the block pointed at by the other reference, before the self reference. This block, to my tests is neither compressed data, MFT, ffna, or ATEXDXT* (The four types of data you recognize on sight).


Again, it's good to look at how the executable handles it, but keep in mind that editors may do all of this differenly. Also, don't rely to heavily on what coders do ;) IMHO ...

The last block you talk about (before the self reference), you don't think this may be hashed filenames? We should look at this closely.

It's interesting that you mention GZip. But I thought GZip wasn't that good in compressing. Hmm.

Well, we'll get there eventually! Good work! Hope to hear your thoughts! Or of anyone who knows something we don't, of course. :)


--SilentAvenger I'm sorry it came out so list-like, I was in a hurry at the time.

Well, about ffna's, I think they are directories of sorts. See, I have downloaded the entire Gw.dat possible from the main menu (You know, how it sits there and downloads), came out about 200mb, and some ffna entries, except for having a self entry as most of them do, also had a bunch of other reference entries. My hunch is, as this archive format supports dynamic writing of content, and not only reading, is that they pre-allocate 2 blocks per ffna, so they can grow the listing if needed. I must say I have not found any ffna's with less than a 512 length, or, come to think of it, any block. And the blocks seem to be aligned on 512 jumps, now that I go check. So, the block size might be 512, and not 256.

About decompression... We could try a bunch, just copy the compressed stuff out of the file, ignoring the constant bytes, and try various things on them.

Also, there seem to be various types of ffnas, some with file entries, some with blocks of data (without the suffix the compressed ones have).

About breaking off the download, being able to see the file at parts of being downloaded has helped me alot, for example, the fact that the number at 0x10 is an MFT offset. If you have different points in the file development, it helps. I also run some statistical analysis using tools I write in C++ on the files, and more test subjects is less errors.


--Republicola A few notes: I don't think you can get a "complete" archive. You download files as you need them, and when idle (in main menu, but maybe in other situations). But even if you left the game in the main menu for a long time, it is unlikely that you would get a special completed archive because they are regularly adding new files and changing old ones. The archive when there is nothing new to download could differ as little as one bit from an archive still in the middle of receiving files.

Gzip is a file format. The gzip system uses compression from zlib. Keep in mind that compression quality is very subjective; both small data size *and* speedy decompression are desireable in a compression system. I got this interesting error message from someone else (it occured when loading Gates of Kryta map, I believe): (2) File 0x8381 stream 0x1 is corrupt (1) Map file '0x008381' failed to load. Attempting to re-bloat. The "bloat" part seems to be a name for the decompression algorithm. Zlib calls it inflate, and I don't know about other systems. This is highly speculative, of course.

Finally, I suspect that file names are never used by GW. See how it refers to the map file with a number in the error message? I also remember noticing a lack of file name strings when I was running the game in my debugger a while ago. They could easily be stripped out of the process by the tools the designers use. This would mean no hashing and less space (of constant size) needed to store a file reference.


--Mr.Mouse 07:32, 3 Jun 2005 (EDT) We may introduce the rule to add new comments to the top, saves scrolling down. For now, I'll post below.

A few notes: I don't think you can get a "complete" archive. You download files as you need them, and when idle (in main menu, but maybe in other situations). But even if you left the game in the main menu for a long time, it is unlikely that you would get a special completed archive because they are regularly adding new files and changing old ones. The archive when there is nothing new to download could differ as little as one bit from an archive still in the middle of receiving files.

It doesn't matter if they change files constantly, they will still need a format of the "new" archive that the executable can read. Chances are very high therefore that the format of the game resource archive will always be the same when fully loaded. And that is what the game will need: a fully downloaded archive.

Well, it may be indeed that no filenames are used in the archive. Nevertheless, we should not dismiss it yet until the purpose of each bit in the archive is identified ;)

I have found that int the total file (256 mb) there are three mft tables. They also point to three pre-self reference tables. These last are exactly the same, though, it seems, while the mft tables do have some difference (just the pointers). Seems redundant info. Also, after the three mft tables comes another large chunk of (compressed) data. Perhaps that is 1. junk 2. filenames compressed??

I have found some refs to compression in the exe:

  • FCArchive
  • CmpHuff --> would probably mean Huffman encoding? [2]

Also, it seems there are markers for different types of files that compress/decompress differently.


--SilentAvenger I Checked the EXE, and there seems to be a bunch of files called CmpIO, CmpHuff and CmpDict, and a function called CmpDecompress, which GW calls. I have managed to locate the function GW uses to decompress, but havent messed around with it. To find it, Search the strings for "Cmp", there is an error message form FcArchive regarding problems extracting, follow it, and you get to a proc, which contains a call to the function that does the actual decompression.


--Republicola I don't understand your reasoning for using a complete archive. Even if the format is different, almost all the archives people have are not complete, so that format would be less useful. I think it will be easier if we are working with different archives--it's like looking at an object from multiple view points to determine its 3D structure.

My archive is a bit over 1 GB, and there were eight Mft sections when I last checked. Does that number increase roughly in a linear manner with archive size then? There might be a max size, which would make sense for a hash table.

Regarding Huffman encoding, it seems unlikely that it would be used as the standard compression algorithm throughout the archive. It is really bad for compression of random binary data. It is only useful for transmitting text that you know certain things about (that it's English, for example). What that article doesn't mention is that you also need to store the table of binary sequences along with the compressed data. There are standard tables for specific applications (like English text) that make it more efficient and useful because you don't have to include a table with the data. Maybe it is used just to compress text or something though.


--SilentAvenger I have updated the structure with information I found using statistical analysis of about 20 archives. Also, I must congratulate Republicola on extracting an ATEX entry! (Nothing much to look at)

Regarding the counters, did some more stat analysis. They are completely sequential, and start at 11 or so. Except for 1, they are in a contigous block, some of them in the hash table thing, and some in the MFT table entries. The hash table sometimes has 2 entries for the same counter, the second one's first uint32 much smaller than the first's.

Also, files in the MFT with sequential counters, seem to have the flags 3,16 for the first one, and 3,16,17,25 for the second one. Except for rare occasions of a 3,16,17,24 file, with no pair.


--Republicola

finished.png

So here is the image I extracted from one of the ATEX sections in my archive. It was the second ATEX section, which I used because the first had 'DXTA', which isn't an actual compression format. This one was 'DXT5', though it isn't exactly that anyway. For reference, the data is here: http://rep.undev.org/gw/07537E00.dat

The image data starts at 0x1c. The only obvious element of the header is the width and height, which are the first two shorts after 'ATEXDXT5'. The image data works as normal DXT5 [3] with a few differences. First, only the alpha part is there. Normally, it is organized into 4x4 pixel blocks (texels) with eight bytes of alpha information and eight bytes of color information. Here, the color information is missing, so texels are only eight bytes. To view the image, I added a DDS header and put in eight byte blocks of zeros every eight bytes throughout the image data.

With that done, the image still wasn't right, but it was obvious that it was missing several texels, which shift all the following texels left. The missing texels were the ones in the four corners of the image (all zero data, since they are completely black). I am pretty sure that for extra compression, they leave out these texels and give their offsets elsewhere (only about two bytes to store the offset, rather than keeping the eight bytes of empty image data). If I counted correctly, the offsets (in texels, not bytes) of zero texels for that image are 0, 7, 56, 63 or 0, 6, 54, 60, depending on whether you add one for each previous texel. I haven't examined the data very carefully for these values yet.

I don't know how to get the right lower mipmap levels yet. What is interesting is that assuming normal DXT5 data (color included and no missing texels), the sizes work out perfectly for a 32x32 image with 6 mipmap levels if you start the image data at 0x14. Maybe GW needs to trick DirectX into thinking it's dealing with real DXT5 data at some point.


--Mr.Mouse Nice puzzling! Keep it up,I'd say! We are making progress here!


--SilentAvenger No kidding eh? Well. More good news. I can now extract files by hand via ollydbg. I have isolated some of the functions related to decompression.

Files are stored in streams. I have yet to find out how that works, probably with that weird hash table thing. The buffer passed to decompression is the exact contents of the block, minus the last 8 bytes.

If someone in here knows ASM,

004BCB72 - Call to allocate the buffer. ECX contains uncompressed file size (last uint32 of compressed file data). Returns allocated space in EAX.

004BCBBA - Call to decompress the file block. EDX contains compressed file size. EBX is the compressed buffer pointer. ECX contains destination buffer.

inside decompressing function:

004D8EBA - Call that actually decompresses. ECX contains destination buffer.

after that call completes you can freely grab the decompressed file from the EXE memory. I've already looked at the prefs file and the gui file for GW.

TODO - figure out the hash function, figure out about this "stream" stuff.


--SilentAvenger Update: Using the ASM skills of xttocs and republicola, we managed to call some functions within GW at will, and this has allowed us to extract a bunch of files, but not all of them. Update: Now I have managed to extract all 9104 files from my archive. seems like bit 3 really is a compression flag, but sometimes you have comrpessed files without that set, seems like leftovers, as there seem to be multiple versions of the same file.

FFNAs are game data! so are ATEX! Some of the compressed files are ATEX or FFNA, therefore, these are *not* part of the archive. What we need to find out now is how GW knows which files to get.

On a side note, GW has one pretty funny language, if we manage to re-compress and insert the prefs file, we could try finding out which one it is.


--Mr.Mouse Has the archive format been investigated enough to move to the main GRAF index?


--Nicoli_s well we have an extractor partly working but without filenames, so you'd have to talk to republicola and silentavenger about that, plus, it hooks the dll instead of reading the .dat file straight


--xttocs Did you manage to call the GW routines from the dll without crashing?


--Harrowed Someone wiped the data. *sigh* Restored.


--Mr.Mouse : Thanks. Damn children, probably. At least: of that intellect. ;)


-- Saibot : Is it possible to get the source for the current extractor somewhere, I might be able to give a hand, but starting everything from scratch to contribute is not something that appeals to me ;-)


--Nicoli_s well i have no problem with it, but what we need to do is to figure out how to actually read the .dat file, instead of hooking the exe as we currently do, but ill talk to republicola


-- Saibot : Any news?


-- MqsTout : It looks like GW stores a lot of tiny files by the way it downloads so many thousands. Does it actually have coherent whole models, or does it use pieces it then reconstructs together?


--Harrowed No updates in a while.. Anyone got anything? If we can export, shoudln't we be able to import... But would this update realtime cfg's etc.. IE updating Clipping range etc.


-- Ral : Still no news?


--Otac0n Don't mean to pester, but does anyone know if the files end at (offset + size) or at (offset + size + 4)?

Just wondering, because it seems that the "ATEX" and "ffna" are filetypes, and not necessarily counted in the file size.


--AzuiSleet: This almost seems like a lost cause, but there's some people left on irc.wc3campagins.com #hakhak I was also able to extract sounds from the archive: http://azu.brokenedge.net/000000000B031600.dat.mp3 the keyword is to look for LAME in the file (lame mp3 encoding)


--Mqstout: Has there been any further progress on this made? Something usable?


---Ricky26:

Hey, you have the cracker of B&W2's STUFF file here, and ready to hack... *readies axe*

Introductions made:

0tac0n: Yes, ATEX and ffna are counted in the size.

Ive noticed that FileMon says it reads 384 bytes at the start of the Mft... That explains those extra indexes away. If we can find the 384 (Is it hardcoded?) .... *mutters to self*

Shall I whip up some C# for analysis? =)


Things to look for:

  • AMAT - Material
  • GRMT
  • DX9S - Shader


Also, I think that the encoding is infact Huffman ( and Im working some C# magic now ). Look at a Zip file and GW.DAT, similarities?
My Guess is this:

  • The server has a bunch of Huffman files (or a complete GW.DAT file).
  • It sends the data ENCODED over the network. (Saves bandwidth).
  • The client stores it as huffman. (Saves space).
  • The client decompresses it at runtime. (Bloat is NCSoft made?)


If this is true then the following may be:

Are FFNAs the Huffman Tree? (Each FFNA could be a file...?)


Actually, what if they just made one Huffman Tree and everything uses that. (It could be hidden in the EXE). Any thoughts on filenames? Or File-References.


Day 2: (lol)

Anyways, I have some good news (for home made decompression).
Their compression looks like re-arranged Zip Format:

  • In flag theory 2: 0x08 = Decompressed (Same as in a ZIP). (0x00 = STORED Hmm.... I smell some ZIPyness) This _IS_ the first short... The second short is flags (the same as in ZIP).
  • Compression has flags 0x0102 and 0x0304 (Dir & File headers in zip)
  • CRC is an INT in zip.
  • Zips start with "PK" (0x0102) and these start with 0x???? 0x0102



Im gonna add these specs to the main page as Theory 2. =P
We almost know enough to write non-compressed files into the FcArchive.... =)
--- On another note, is there anyone here still alive?

THERE ARE FILENAMES
Ive found conclusive evidence that there are filenames...
The folders start with /. E.g. /Art/Ui/ (actual dir).
Now I need to find them in the .DAT =P


Mqstout:I'm still alive here, eagerly awaiting your answers. Sadly I don't know my way around. Last file hacking I did was decomposing saved game formats for that ancient DOS game Drakkhen. I just look forward to useful... uses (in extraction)

-- Ricky

Does anyone know how SilentAvenger ran the decompression code?

=(


--- Ricky26

"Files are stored in streams. I have yet to find out how that works, probably with that weird hash table thing. The buffer passed to decompression is the exact contents of the block, minus the last 8 bytes."

Thats because the last 8 bytes is "new flags" and the decompressed size (int32). ¬_¬


mqstout: I don't know if it'd be helpful to look at the files as they're stored on the CDs the game shipped with? The .dat file is spread across two disks (both for the original and the expansion) and diffing them or looking at what's around the break might be helpful?


--Otac0n I definitely have made some advancements, using a tool I created available at http://67.67.35.17/GWDat.zip

Written in C#.

This has lead me to believe that all of the textures listed as "DXTA" are actually DXT5 compressed. These are probably game textures, since they have a file ID.

If we can get some work done on the header of all these files, we should be able to export most of the game's textures.

Update May 22nd:

I have re-coded alot of the GWDat C# project. Grab it again if you want it. It should help out in delving arround alot.



Taylor Mouse : I read in anothor forum that you can decompress the content of the dat file using gw.exe -image. This decompresses all files in the dat file.

User:Ricky26:

0tacon: I was doing the same thing! A C# project... But I was also doing some binary stuff in a C++ one...

You use XFire? ;) (ricky26)

@Taylor Mouse: Thanks for the tipoff.... Unfortunately it decompresses INTO the dat. ='(

Update: I think that -image creates the "clean" archives that you see on the packaged disks!

Update: -image compresses stuffs =(...

???: -image is said to only decompress textures according to bad polish translations

xyz: Actually it's said to download all pending updates...

    see: http://gw.gamewikis.org/wiki/Command_line

Also: I can now extract files manually with OllyDbg... (reinventing the wheel! gah!) but not by choice! just as they are requested =(.

I think that there has to be some kind of executable code in the dat, script or something, because "-image" isnt in the exe.

=)

There may be executables in the dat, but the -image (along with all the other command line switches) IS in the .exe, without the "-" switch. If you look for the "push" assembly commands in the exec, you will find references to "params.cpp" and "cmd.cpp" and "HcmdParser". These routines parse the options out. The routines start at 40B780 and 40B8D0 in the current (as of June 5th) version of the exec.

Decompression Routine

The routine starts at:

508C70

in my version of the exec. The above poster is correct in that ecx contains the decompressed data.

Correction:

Sometimes ecx contains the buffer, but it is sometimes in edx as well.


--Otac0n

Posting addresses from the exe is useless. Every recompile/relink is likely to jumble these around.

When I put a breakpoint on the routine that SilentAvenger posted, it only breaks when decompressing NEW files. AKA only if the file was just now downloaded. So, it may be just a preliminary decompression.


--Ricky26

Is there an IM we can use, this wiki is getting tedious...?

Also, I searched the EXE with a hex editor... So maybe I had it on UNICODE... Oops =)

I can call compression from C++, but I get "Cannot access memory at" errors... =)


--Otac0n

We could IRC.

I'm gonna try to register a channel on GameSurge. "#GWDat"

Join up. I'm gonna log the channel and create a PHP script to show the logs from my webserver.


--palisade

July 15, 2006 - I went to the irc server and that channel was empty. I've just started getting into breaking this format, so far I'm just reading out the headers and verifying them, in C++. I noticed there are dead padded areas not mentioned in the WIP spec, and the description of the FFNA table is missing. I read DAT header, check the CRC, then read FFNA table, then skip to and read the MFT header. Then making sure to read the other headers and skip the empty entries before reading the MFT itself. That's the extent I've gotten to tonight; I plan to try to implement a texture loader, write up a decompressor for other blocks, and hopefully mesh extracter.

What got me interested in doing this is that I found this in the GW log today: (2) Linked item specifies a custom color but uses a model file f7c0 that does not define an armor texture.

July 16, 2006 - Update: I can extract and view the .DDS now. Only 20 ATEX files in the entire archive? That seems odd, and they're very corrupted. Ah, I was skipping DXTA, there are 31 entries now.

In response to what Ricky wrote regarding decompression: `Zips start with "PK" (0x0102) and these start with 0x???? 0x0102`

I think the first four bytes represent either a lookup into a table of names and/or other 4 byte identifiers, or it is the four byte identifier itself. I searched for the f7c0 that was listed in my gw log as a model with a bad texture assignment but I was unable to find it. (got pruned?)

Scratch that theory, there are too many redundant instances of the "identifier", so it must be a type id of some sort? E.g. There are 838 instances of 0x1c1d. Highest number in the identifier I've seen is 0xfb in the high byte and 0xe6 in the low byte.

Anyone else still working on this?


another one - a developer said in an interview, that every file has its own version number. So maybe these numbers are the version. Btw. You wrote you can extract some files. Is it possible that you release something like a version 0.00001 pre alpha programm for all the stupid guys like me, just needing some files? =)


I hope someone uses this weekend's Nightfall preview event to good. For example, the first file it downloads once you're in the client and go to Random Arenas is >3 megs. I'm guessing it's a music file. For me, it was at 2061 files remaining for the entire first ~4 megs. -mqstout

-- Ricky26:

We've lost SilentAvenger's code. We can still get files out with debuggers, (copy and paste memory sections hehe), like ollydbg, but we have no software for it.

I need a genius at code injection, (my simple methods dont work! D: )!

And a version, you say? No CRC? If there is no CRC, this could be verrry good :P.

-- Unregistered

It seems like someone managed to get some models out of the game file. The textures should be in .dds file format. A friend of mine found these pics randomly online. Url unknown. 349409e8.jpg th_Untitled8.jpg

--

Just to let you guys know I'm giving a try at reversing GW stuff. I'm fairly experienced in this field (check my blog: http://www.yaz0r.net). Right now I'm reversing from assembly the .dat access routines and decompression routines. I hope to have something to show soon... Yaz0r 13:17, 7 August 2006 (EDT)

Update: reversed the checksum algo. It's very simple, I'll post the code for it soon. Yaz0r 15:58, 7 August 2006 (EDT)

What are you using to decompile the code?

You should look into the freeware version of IDA for this. I can confirm that the crc used is the same as the one in ZIP, but with a different loockup table. Yaz0r 16:35, 7 August 2006 (EDT)

--Ricky26: Aha! A new person! Im working with IDA... It is very useful. I have largely-commented bits of assembly, I work towards re-using the binary code. (I'm not great at understanding assembly!) But as I have said, injecting doesn't want to work. Looking forward to that code!

Out of interest here is a file: (Presumed to be CategorySettings.ini)

;*****************************************************************************
;*
;*   CategorySettings.ini
;*   
;*   By James Boer (02/20/2006)
;*
;***/

;*===========================================================================
;*  Example:
;*  [category name] - WARNING: cannot be more than 7 lower-case characters
;*  Volume=<-10000, 0>          Main volume attenuation in millibels
;*  Effects=<-10000, 0>         Effects wet mix attenuation in millibels
;*  MinLooping=<0, ???>         Max num looping sounds at minimum quality level
;*  MaxLooping=<0, ???>         Max num looping sounds at maximum quality level
;*  MinNonLooping=<0, ???>   Max num nonlooping sounds at minimum quality level
;*  MaxNonLooping=<0, ???>  Max num nonlooping sounds at maximum quality level
;*
;*  The max number of simultaneous sounds are interpolated between the values
;*  of the minimum and maximum quality values, based on current quality value.
;*===========================================================================

[default]
Volume=-100
Effects=-10000
MinLooping=2
MaxLooping=20
MinNonLooping=2
MaxNonLooping=20

[skillb]
Volume=-350
Effects=0
MinLooping=3
MaxLooping=10
MinNonLooping=5
MaxNonLooping=10

[skills]
Volume=-350
Effects=0
MinLooping=3
MaxLooping=10
MinNonLooping=5
MaxNonLooping=10

[skillx]
Volume=-100
Effects=0
MinLooping=4
MaxLooping=10
MinNonLooping=6
MaxNonLooping=10

[woosh]
Volume=-200
Effects=0
MinLooping=2
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[plyvoc]
Volume=-50
Effects=0
MinLooping=2
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[monvoc]
Volume= 0
Effects=0
MinLooping=2
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[arrhit]
Volume=-200
Effects=0
MinLooping=1
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[axehit]
Volume=-250
Effects=0
MinLooping=1
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[daghit]
Volume=-200
Effects=0
MinLooping=1
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[hamhit]
Volume=-500
Effects=0
MinLooping=1
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[maghit]
Volume=-200
Effects=0
MinLooping=1
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[swohit]
Volume=-550
Effects=0
MinLooping=1
MaxLooping=10
MinNonLooping=2
MaxNonLooping=10

[effects]
Volume=-50
Effects=0
MinLooping=4
MaxLooping=8
MinNonLooping=4
MaxNonlooping=20

[feet]
Volume=-50
Effects=0
MinLooping=2
MaxLooping=6
MinNonLooping=2
MaxNonLooping=6

[smprop]
Volume=-300
Effects=0
MinLooping=1
MaxLooping=6
MinNonLooping=1
MaxNonLooping=12

[mdprop]
Volume=-300
Effects=0
MinLooping=1
MaxLooping=6
MinNonLooping=1
MaxNonLooping=12

[lgprop]
Volume=-200
Effects=0
MinLooping=1
MaxLooping=6
MinNonLooping=1
MaxNonLooping=12

[ambloop]
Volume=-200
Effects=-10000
MinLooping=2
MaxLooping=12
MinNonLooping=2
MaxNonLooping=12

[ambone]
Volume=-200
Effects=-10000
MinLooping=1
MaxLooping=8
MinNonLooping=1
MaxNonLooping=8

[speech]
Volume=0
Effects=0
MinLooping=4
MaxLooping=4
MinNonLooping=4
MaxNonlooping=4

[ui]
Volume=0
Effects=-10000
MinLooping=2
MaxLooping=4
MinNonLooping=8
MaxNonlooping=12







... Also, I'm gonna embark on the slow, hard, de-compiling of the cmpDecompress... Got the begining of the function done.

I've been reversing large parts of cmpDecompress. Right now, I'm in the middle of the huffman code. This is boring as hell.... -_- Yaz0r 15:49, 13 August 2006 (EDT)

I've found a way to get those. I'd be willing to bet whoever got the textures/models posted above used a similar method. The trick was to capture data on its way to the GPU, recording everything that gets rendered. If you'd like to have a look at the original DDS files, I have access to however many you may need. GW also appears to build certain textures (the hair and chest to be specific) by examining the avatar's hair type and chest armor, then sticking the two bits (colored by the time they hit the GPU, I can't vouch as to how it does this specifically, although similar things are noted in a few other hooked scenes) into one texture. If something isn't present it shows up as a black area on the skin. Models are also multi-part, if anyone's wondering. This is pretty obvious when the occasional hang occurs, as the head/any exposed pieces of skin will appear, followed shortly thereafter by the rest of the character's clothing and hair. I would assume this happens when GW builds an avatar's skin. -OneOneSeven


Liquid:

"I need a genius at code injection, (my simple methods dont work! D: )!"

Hey I just read this and I don't know if you still need it but I can do code injection in ASM. I've just done it before on GW in the process of creating an application I call GW Multi-Instancer, it intercepts the CreateMutexA and RegQueryValueW functions to make the game ignore other instances and the locations it's running from. That application is fully working, it would be quite simple for me to perform any similar code injection on other functions.


Getting the models from the game.

Getting ou the models and the textures, there is a trick to do this. But it is only for tudying - of course -.

Download the 3D Ripper DX from "http://www.deep-shadows.com/hax/3DRipperDX.htm" . Start the game and press the "capture screen" defined in the application. Import it with 3DSMax and you can study the models. You do need a licensed version of 3DSMax 4-8 - of course -.

All textures that are taken are .dds format.

The negative side of this is that you need to play the game to get the models.

But hey, this is better then nothing :)

-- Ricky26:

I got to the huffman in cmpDecompress, afaik.

It looks like they push the whole table onto the stack!


Is there still anyone working on this file? --Blackbeard 05:08, 24 October 2006 (EDT)


I emailed a guy who reverse engineers lots of game clients and programs (like Steam games, gamespy games, and some Ventrilo stuff). He is very smart and has helped me with some personal projects in the past. For example, he helped me port the login process for gamespy based games (like bf2) from C over to php and vb. Also, he helped me port an online cd key checker (for any gamespy based game, particularily bf2) from C to php as well. Here is his website with all his projects, a very very nice site for tons of tools for games he has reversed: "http://aluigi.altervista.org". I will keep you informed if he decides to work on the Guild Wars .dat extraction project. He should post all the work he does on his page, so you could get it there too if he decides to do it. -soadlink


--palisade (nov 16th 2006) yea sorry about that, i've been working alot lately and haven't had a chance to dust off my old code. it doesn't actually do anything but maybe it will help someone.

http://www.rm-f.net/~palisade/gwextract.zip

Tried your tool but it gives me an error. However, my GW.DAT file is definitely not bad, since I'm playing GW just fine with it:
@ 0x00000000 -- Reading DAT header. . . BEGIN DAT HEADER----------------------------- DAT Signature  : 3AN DAT Header Size: 32 bytes DAT Sector Size: 512 bytes DAT Header CRC : 0x4CCBAD70 DAT MFT Offset : 8c1a2600:00000000 DAT MFT Size  : 2873304 bytes DAT Flags  : 0 END DAT HEADER------------------------------- >>> DAT header checksum passed! @ 0x00000020 -- Skipping to the FFNA sector. . . @ 0x00000200 -- Skipped. BEGIN FFNA HEADER---------------------------- FFNA Signature : ffna FFNA Unknown1  : 1027074 bytes FFNA Unknown2  : 3072 bytes FFNA Entries  : 256 FFNA Unknown3  : 0 bytes FFNA Unknown4  : 0 bytes END FFNA HEADER------------------------------ @ 0x00001200 -- Reading MFT header. . . BEGIN MFT HEADER----------------------------- MFT Signature  : ñoı MFT File Count : 2209144744 MFT Unknown2  : 4275768184 bytes MFT Entries  : 1248716975 MFT Unknown3  : 1750269727 bytes MFT Unknown4  : 3589169441 bytes END MFT HEADER------------------------------- *** Invalid GW.DAT, main file table signature bad.
This seems to be because my GW.DAT is >2gb. Changing some fseek() calls to _fseeki64() seems to help there... still investigating
--Darkstar (Nov 17, 2006)

I was reading some stuff on "World of Warcraft" today and discovered that Mike O'Brien was the one who created the .MPQ file format for Blizzard. So that got me thinking that the gw.dat probably has a very similar layout to a .mpq.

See here:

[url]http://www.zezula.net/[/url]

Click on the "MPQ format" section.

Let me know if this helps. As I mentioned above, I'll bet those two file formats are similar. He also has an algorithm for decompression there as well.

BD

Made some progress on this. See http://www.yaz0r.net/forum/index.php?topic=103.0

BD


http://www.directsong.com/zip/GuildWars.txt this is an INI file from the DirectSong plugin that changes what music plays in certain towns and outposts in the game. It might be useful for determining some filenames in the .dat, perhaps? (Slightly associated thread: http://www.guildwarsguru.com/forum/showthread.php?p=2637560#post2637560). --Mqstout 11:48, 17 March 2007 (EDT) Gaile posted some documentation for that ini file. Unfortunately it doesn't look as useful now -- but maybe! http://www.guildwarsguru.com/forum/showpost.php?p=2669207&postcount=5 --Mqstout 18:09, 27 March 2007 (EDT)

Disassembly

If anyone knows Assembly, you can unassemble the .dat file, I did it a while ago, but I do not know ASM. Use the nASM unassembler (ndisasm.exe) in a command line and output it to a file (add >> [filename] at the end of the command) WARNING: size of the output is scary and the unassembling takes a long time. — Preceding unsigned comment added by 173.16.198.187 (talkcontribs) 00:30, 26 November 2011

Unless the structure of the DAT file has substantially changed since the above work was done (which I doubt), disassembling it would just result in meaningless gibberish since it isn't an executable file, but a custom archive format. ディノ千?!? · ☎ Dinoguy1000 21:33, 25 November 2011 (EST)