DDSopt

From OBGE - Oblivion Graphics Extender

Jump to: navigation, search

DDSopt is a tool which manages and recompressed DDS-files. It has been forked off NIFopt and both programm share the same underlying technology regarding DDSs and BSAs. Both programs treat BSAs exactly like directories, files inside can be adressed easily.

Contents

Downloads

It can be downloaded here: DDSopt v0.7.3 (04.02.2012) and here: DDSopt v0.7.3 (04.02.2012) (this is now release-candidate)

Source-code is available under the MPL on github.

Installation

Just download the archive, extract it where it suits you, and make sure you have the necessary runtimes for your Operating System installed. The archive contains two links to the locations where you can download the version which is right for you.

It's recommendable to install DDSopt into a sub-directory of Oblivion, something like "G:\Oblivion\DDSopt\", so you can easily just delete that folder to uninstall it.

Quickstart:

Optimize a BSA for size and process all DDSs at the same time:

browse for the input file
This will load the BSA into the tree- and file-controls.
browse for the output file (type the name)
This will define where to write to.
click "Convert"
This means DDSopt starts processing everything from the input BSA into the new BSA you defined.

That's it. After a while the progress-dialog will close and the new BSA is ready to use.

What exactly does it do?

It can convert and optimize textures, both in size and quality. The utilized compressor is also built-in in the Shader-Developer of OBGE, for which it has been developed initialy. It treats every image, that is of one of the supported image-formats, which resides in a "textures"-folder as textures and won't touch images outside that folder.

Basic modifications (without message):

  • stripping excess mipmap-levels
  • appending missing mipmap-levels

In addition it can compress uncompressed textures (from any format):

  • high-quality custom DXT-compressor built-in
  • high-quality custom 565 built-in
  • automatic selection of optimal DXT-format
  • automatic grey-scaling
  • stripping of white alpha-channels (alpha is white by default in hardware)
  • stripping of black color-channels (color is black by default in hardware)

As an alternative to common compressors it also offers high-quality mipmap-creation with the option to do gamma-correct mipping. Tangent-space normal-maps can be comrpressed in a high-precision variation as well.

As the tool can not assume the alpha-channel contains actual transparency-information you can drive it on a per-file basis:

  • indicate you want to compress an otherwise unidentifyable normal-map, append the suffix "_xyz", like "mytexture_xyz.png"; it'll become "mytexture_n.dds"
  • indicate you have a diffuse-channel as well, append the suffix "_xyzd", like "mytexture_xyzd.png"; it'll become "mytexture_n.dds"
  • indicate you want to compress an otherwise unidentifyable color-map with correlated alpha, append the suffix "_rgba", like "mytexture_rgba.png"; it'll become "mytexture.dds", and apply transparency-related optimizations

If it's not unmistakenly possible to derive knowledge about the alpha-channel it'll always be treated as uncorrelated data and will not affect the color/normal-map.

Walk-through

All elements have tool-tips which hint at the meaning, or occasionally tell you the optimal or default value. Most fields auto-remember their values and are persistant between program-runs.

The Browser

DDSopt archive browser

Description of the UI-elements:

Source
This is the directory/BSA which DDSopt will scan recursively and display in the tree-window below. BSAs in directories will be scanned recursively. Click "Browse" to get the popup. To operate on folders, just navigate to your target-directory and press the button "Use folder".
Hint: The field automatically remembers the last location you navigated and restores the location the next time you start DDSopt.
Directory-tree
Displays all sub-directories/BSAs of the selected source & destinations overlayed. You can open and close the free-branches. Once you active a branch by clicking on the name you get a listing of the files it contains. The tree reflects the relation between source and destination via coloring. The color-scheme is:
  • "green" - new files in the directory to add to the destination
  • "grey" - exist only in the destination
  • "black" - files in a directory will be skipped or overwritten, or a directory contains both new and destination-only files
When you double-click on a branch it will be loaded into the "source".
Filter
Allows you to filter the list of files on a directory by a wildcard. There are a few predefined filters in the drop-down, namely all texture-suffixes known from Oblivion and Skyrim. When "Apply" is clicked the filter is validated and applied to the file-list if correct.
Toolbar
You have a few tools available here which relate to the list of files below:
  • The first button selects all visible files
  • The second button deselects all visible files
  • With the setting "Show recursive" you can make DDSopt display not only the files which directly reside in a directory, but also all files in sub-directories.
File-list
Here you manage the selection of files. The checkbox indicates if a file will be copied to the destination. Files are automatically filtered by the active filter. The color-coding is identical to the directory-tree, plus:
  • "strike-through" - files will be skipped because of some setting that you've set
  • "italic" - files are compressed in the source
  • "blue" - files will only be copied because of some setting that you've set
When you double-click on a file it will be loaded into the preview.
Destination
Here you define the location of the destination directory/BSA.
Hint: The field automatically remembers the last selected location and restores the location the next time you start DDSopt.

Description of the Menu:

Game
This will define the BSA-version for a specific game, in case a BSA will be created in the destination. "Automatic" will always duplicate the incoming BSA-version, the others will force a specific game-version. You can transcode from Oblivion-BSAs to Skyrm-BSAs this way.
Behave
Here you can adjust what and how something will be done to images:
  • "Don't touch HDR" stops DDSopt from converting HDRs to 8bit.
  • "Produce partial derivative tangent-space normal-maps" is explained here.
  • "Raise normal-map steepness each mip-level" recovers some normal-map contrast (curvature) when it becomes washed out from mip-map down-scaling.
  • "Gamma-correct mip-map filtering color-maps" makes DDSopt to resample mip-levels in gamma-space.
  • "Ignore borders on optimizing" causes the optimizer not to consider/look at border regions (of the configured size) of the textures. This allows DDSopt to optimize textures which have an obvious artifacts from rescaling. It is also prevalent in the vanilla Skyrim-textures.
  • "Compress all kown texture-formats" enables conversion from fe. PNGs to DDS-DXT
  • "Re-compress and re-mip all textures" forces DDS-files to be revisited, optimizations like channel-reductions only happen when this is on.
Ignore
Allows you to skip processing on "every known file-type". This puts DDSopt essentially into BSAopt-mode.
"Don't process any already processed files" recognizes already treated files and skips those. That is especially helpfull when you want to make down-scaled versions of optimized textures, as those won't be re-evaluated again.
Compression
Defines the compression-strength. "0" turns compression off. "9" is the normal maximum of zlib. "10 (quick)" will do a quick search for a better compression, "10 (maximum)" will do an exhausive search for the best compression and usually takes longer. By default compression will be turned off when the compression is not very high (>95%), you can force the compression regardless the efficiency via "Forced".
Settings
With these settings you allow DDSopt to skip files, "Skip existing files" will never overwrite a file in the destination (BSAs are considered directories), "Skip older files" compares the timestamps of the source and the destination and overwrite only if the source is newer (BSAs define the timestamps of their contents), "Skip hidden files" will make DDSopt to skip all hidden files. Changing the settings will automatically reflect in the file-list.
You can also skip the sanity-check for BSAs if you want to recover a broken BSA with the "Skip hash-check" setting.
And if some files can't be decompressed and you don't want a dialog reporting it, you can select "Skip broken files".
If you want a transscript of all details and operations, then turn on "Write logfile". You'll find the logfile at the location of your destination + ".log".

When you made all your adjustments (which minimally is just selecting the source/destination-location) you press "Convert" on the lower right corner and DDSopt will start to copy files "over".

If everything goes fine, you'll either have a directory full of extracted and processed files, or a BSA with the contents you selected previously.

The Preview

DDSopt texture preview

Description of the UI-elements:

Path
Here you see the complete path of the texture you double-clicked on.
Surface
Textures can have multiple independent and interdependent channels. You can select which set of channels you want to see in the preview.
Preview
This shows a render of the texture. Cube-maps will be rendered as cross-probes. Normal-maps will be normalized before display.
Infos
Tells you a bit about the texture. The purpose(s) for the game, the dimensions and mip-levels as well as the format and compression the texture is saved in.
The following use-cases are known (for Oblivion and Skyrim):
  • in a "menu"-folder: User-interface element, alpha is element-mask
  • in a "landscapelod"- or "terrain"-folder: World-space normal-map for terrain, no alpha
  • with a "_n"-suffix: Tangent-space normal-map, alpha is specularity
  • with a "_msn"-suffix: Model-space normal-map, no alpha
  • with a "_g"-, "_glow"- or "_emit"-suffix: Glow-map, alpha is a mask
  • with a "_hh"-suffix: Gloss-map for hair, no alpha
  • with a "_hl"-suffix: Detail-map for hair, alpha is opacity
  • with a "_m"-suffix: Reflectivity-map for light-sources, no alpha
  • with a "_em"- or "_envmap"-suffix: Reflectivity-map for environment-maps, no alpha
  • with a "_e"-suffix: Environment-map (some are planar, some are cube-maps), no alpha
  • with a "_b"- or "_bl"-suffix: Backlight-map, no alpha
  • with a "_s"-suffix: Specularity-map for skins, no alpha
  • with a "_sk"-suffix: Tone-map for skins, no alpha
  • with a "_p"-suffix: Parallax-map, no alpha
  • with a "_d"-suffix: Diffuse-map, alpha is opacity
  • with a "_h"-suffix: Haze-map, alpha is unknown
All others are Color-map, alpha is opacity or parallax
Format & SaveAs
This allows you to quickly save the active image in a format of your choice. Only a minimal set of optimizations is applied.

Constraints

DDSopt conversion constraints

This entire block of settings allows you to finetune the behaviour of/constraints applied to DDSopt. You can set file-formats for specific texture-types and limit the size of the textures.

Description of the UI-elements:

Optimize / Compress
The checkbox of each texture-type tells DDSopt to optimize that type. This is the individual equivalent to the menu option "Re-compress and re-mip all textures". If you turn this option off and a texture is already compressed, or not scheduled for compression, nothing happens.
The drop-down then tell DDSopt to which format to compress when a texture is scheduled for compression (that is basically when it's none of the DXT-formats).
Resolution-limit
After the texture has been optimized and possibly compressed you can cap the resolution of that image. For example when you want to preserve high quality normals, you don't compress them, you make them smaller instead. So when the texture arrives here you can drop it's size to say 1024x1024 instead. There is a cap config for both compressed and uncompressed textures available.
Size-limit
...

The formats DDSopt can read/write are:

  • X8R8G8B8 (color-map + normal-map)
  • A8R8G8B8 (color-map + normal-map)
  • A1R5G5B5 (color-map)
  • X1R5G5B5 (color-map)
  • A4R4G4B4 (color-map + normal-map)
  • X4R4G4B4 (color-map)
  • R8G8B8 (color-map + normal-map)
  • R5G6B5 (color-map + normal-map)
  • A8L8 (color-map)
  • A4L4 (color-map)
  • L8 (color-map)
  • A8 (color-map)
  • A1 (color-map)
  • DXT1 (color-map + normal-map)
  • DXT3 (color-map + normal-map)
  • DXT5 (color-map + normal-map)


The progress-dialog

DDSopt-GUI-Progress.png

Once you start the copying the progress-dialog pops up and informs you about what it is doing. You can pause and cancel the copying, sometime it takes a while till DDSopt finishes a sub-job to react to your button-press.

Below the progress-bars there is a time-estimation where you can see if you need to stop it and continue at a later time.




Customization

DDSopt can read .ini files in which you can specify the contents of a specific texture and configure the kind of compression you want for a specific file. There are three possible .ini DDSopt reads in:

  • DDSopt.ini
  • DDSopt-Oblivion.ini
  • DDSopt-Skyrim.ini

The first is always read in, the other are read in dependent on the game you selected in the menu.

This feature is very practical when textures can not be matched by suffixes (like _n for tangent-space normal-maps). Also, the default behavior for Oblivion is to assume the alpha-channel can be anything, but for Skyrim it assumes per default that the alpha-channel of a color-map is opacity. In this list you can define excatly what the alpha-channel in each file means. Normally DDSopt is smart enough to handle most textures, so use the feature wisely and only as a backup-plan.

Default .ini

This is the default .ini ("DDSopt.ini"):

; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; syntax and compression-formats are:
;
;  subdirectoryoftextures/texturename.png=DXT
;  subdirectoryoftextures/texturename.bmp=BIT
;  subdirectoryoftextures/texturename.dds=RAW
;
; =DXT	-> compress to a suitable DXT-format (DXT1, DXT3, etc.)
; =BIT	-> compress to a suitable bitfield-format (565, 4444, etc.)
; =RAW	-> don't compress
;
; - you don't write the "texture/"-path it will be stripped anyway
; - you can omit the compression-type, then it'll only make DDSopt understand
;   that it's a specific file-type which it oserwise couldn't recognize
 
; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; - mark textures to be tangent-space normal-maps, enables specific compression
;   taking unit-halfsphere property into account
; - compression-format is supported here
[NormalsTangentSpace]
 
; Realistic Water Textures
effects\fxwatertile01_n.dds
effects\fxwatertile02_n.dds
effects\fxwatertileflat_n.dds
water\creek.dds
water\defaultwater.dds
water\lake.dds
water\ocean.dds
water\river.dds
 
; Pure Waters
purewaters\purewatercalm.dds
purewaters\purewaterdefault.dds
purewaters\purewaterstorm.dds
 
; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; - mark textures to be model-space normal-maps, enables specific compression
;   taking unit-sphere property into account
; - compression-format is supported here
[NormalsModelSpace]
 
; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; - mark textures to be color converted
; - compression-format is supported here
[DiffuseColor]
 
; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; - mark textures to be greyscale converted
; - compression-format is supported here
[DiffuseGrey]
 
; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; - mark textures to have an alpha-channel with opacity, enables specific
;   compression taking this into account
; - compression-format isn't supported here
[AlphaOpacity]
 
; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; - mark textures to have an alpha-channel with something custom, disables
;   specific compression only working for opacity
; - compression-format isn't supported here
[AlphaCustom]
 
; Dragons have dissipation-maps
actors\alduin\alduin.dds
actors\dragon\dragon.dds
actors\dragon\dragon_boss.dds
actors\dragon\dragon_earth.dds
actors\dragon\dragon_snow.dds
actors\dragon\dragon_swamp.dds
actors\parthurnax\parthurnax.dds
 
; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; you don't know or don't care to specifically define the contents of a file
[Automatic]

Statistics

Oblivion

in bytes out bytes formatchanges reduction
Oblivion - Textures/ 2070539712 2000347963 - 96.6%
Oblivion - Textures.bsa 1217809056 1126565582 - (975 dupl.) 92.5%

Oblivion DLCs

in bytes out bytes formatchanges reduction
DLCBattlehornCastle/textures/ - - - -
DLCFrostcrag/textures/ - - - -
DLCHorseArmor/textures/ - - - -
DLCOrrery/textures/ - - - -
DLCThievesDen/textures/ - - - -
DLCVileLair/textures/ - - - -
DLCShiveringIsles - Textures/ 438383116 436620561 19 99.6%
DLCShiveringIsles - Textures.bsa 312806750 299962744 19 (24 dupl.) 95.9%

Oblivion Mods

in bytes out bytes formatchanges reduction
Qarl TP 3.0/ 2950718241 2705023429 77 91.7%

Skyrim

in bytes out bytes formatchanges reduction
Skyrim - Textures/ 2383177381 2242262835 3642 94.8%
Skyrim - Textures.bsa 1388866502 1329559634 3642 (6206 dupl.) 95.7%