|
MOD - Creating custom textures for planets 0.97
|
|
| SpaceEngineer | Date: Wednesday, 21.12.2011, 15:52 | Message # 1 |
 Author of Space Engine
Group: Administrators
Russian Federation
Messages: 4800
Status: Offline
| Last update: May 21, 2013 (version 0.97)
CubeMap is a software package to convert the texture of a cylindrical projection to a quadrilateralized spherical cube projection (for short cubemap projection). It supports input textures of any resolution, any number of channels per pixel, 8-bit or 16-bits signed or unsigned integers. The format of the input texture is uncompressed raw data. The format of the output texture is raw, and/or the array of tiles of different levels of detail are in the formats of raw, tga, dds, jpg, tif, png.
Download: CubeMap
Note: the CubeMap tool version is v1.05, so this manual is a bit outdated (it was for v1.03). Read the readme.txt file in the CubeMap's folder for reference about the new features (texture border commands change and Glue utility update).
Files:
CubeMap.exe - the tool to convert the texture of a cylindrical projection in cubemap.
Optimizer.exe - the tool for removing "empty" tiles from cubemap texture prepared with the CubeMap tool.
Glue.exe - the tool for bonding tiles to a single raw-file.
default.ccf - the default configuration file for all utilities.
readme.txt - user manual.
Configuration file and command line
All program parameters are specified in the file default.ccf in the program folder. It is a simple text file containing a set of strings of the form
var value # comment
var is a parameter's name, value is a parameter's value, # char separates comments from value. Comments may be either at the beginning of the line, (in this case the whole line is commented), or at the end of the line after value. Parameter value is available in four forms:
string - character string without quotation marks and spaces;
int - integer number;
float - real number;
bool - binary value: true or 1 for true, false or 0 for false.
All utilities can work with any other configuration file, if you pass it as the first command line parameter:
CubeMap myconfig.ccf
Optimizer myconfig.ccf
Glue myconfig.ccf
In this case, the file default.ccf is ignored. It is recommended to associate the files with the extension .ccf with the program CubeMap.exe, then clicking on the *.ccf file will automatically start CubeMap with the parameters from this file. The *.ccf files can be seen as scripts for the texture processing program CubeMap. The .ccf extension can be replaced with any other, so it can be associate each tool with its file type.
At the command prompt, specify the path to the input file and output folder, then the appropriate options from the config file will be ignored:
CubeMap [myconfig.ccf] [-i InputFile] [-o OutFolder] [-h]
Optimizer [myconfig.ccf] [-o OutFolder] [-h]
Glue [myconfig.ccf] [-i InputFolder] [-o OutFile] [-h]
Option -h displays help on the command line usage.
SpaceEngine's files structure
The path to the input cylindrical texture is specified by InputFile, and should not contain spaces. The path to the output folder for the cubemap texture is given by OutFolder, and also should not contain spaces. The output folder must exist. The six subfolders named neg_x, neg_y, neg_z, pos_x, pos_y, pos_z will be automatically created inside of output folder.
The original cylindrical texture should be in the uncompressed raw format. The raw format is simply a two-dimensional array of pixels, its format is described by the following config options:
InputWidth - width of the input image
InputHeight - height of the input image
InputChannels - number of channels (Grayscale - 1, RGB - 3, RGBA - 4, etc.)
Input16bit - bit-depth: 16 or 8 bits per channel
InputByteSwap - for 16-bit: little endian (MAC format)
InputUnsigned - for 16-bit: unsigned or signed values
InputLatOffset - the global shift in longitude in degrees
The first two parameters are integers, the rest are boolean.
The cubemap texture is a set of files and folders organized as follows:
Code
cubemap ----------- folder that points in the parameter OutFolder
neg_x ----- folder containing the tiles for neg_x face of cube
0_0_0.jpg ----- 1 tile of level 0
1_0_0.jpg - \
1_0_1.jpg --+-- 4 tile of Level 1
1_1_0.jpg - |
1_1_1.jpg - /
2_0_0.jpg - \
2_0_1.jpg - |
2_0_2.jpg - |
2_0_3.jpg --+-- 16 tiles of level 2
2_1_0.jpg - |
2_1_1.jpg - |
2_1_2.jpg - |
2_1_3.jpg - |
.........
level0.raw - \
level1.raw - + - temporary files
level2.raw - |
.........
neg_y ----- folder containing the tiles for neg_x face of cube
neg_z ----- folder containing the tiles for neg_z face of cube
pos_x ----- folder containing the tiles for pos_x face of cube
pos_y ----- folder containing the tiles for pos_y face of cube
pos_z ----- folder containing the tiles for pos_z face of cube
The cubemap texture consists of 6 faces, the names given to them are in accordance with the axes of the coordinate system to which they cross. The coordinates origin is in the center of the planet. The Y axis passes through the poles of the planet, its direction is from bottom to top, i.e. the North Pole is at the center of the pos_y face, the South Pole is at the center of the neg_y face. The X axis passes through from left to right, i.e. the "left" side of the globe is at neg_x face, the "right" side is at pos_x face. The Z axis points towards the observer, i.e. the farthest side of globe is at neg_z face, the nearest side is at pos_z face. At the original cylindrical texture, this corresponds to: the upper boundary is the North Pole, the lower boundary is the South Pole, the left and right boundaries run through the neg_x face vertically, the central meridian runs through the pos_x face vertically. The layout of the cubemap faces is standard:
The resolution of the face is calculated as one quarter of the width of the original texture, rounded to the nearest largest power of 2. I.e., for example, if the resolution of the original texture is 86400 x 43200, then the face resolution will be 32768 x 32768 (32768 = 131072 / 4).
Each face is organized as a set of tiles. The resolution of the tile is determined by the TileWidth parameter and must be no larger than the resolution of the face. For example, if the TileWidth is 512, and the resolution of the face is 32768, then there will be 7 levels of tiles: Log2(32768 / 512) + 1 = 7. Levels are numbered from 0 to 6, the number of level is the first number in the tile's file name, for example: tile 0_0_0.jpg is level 0, tile 6_63_63.jpg is level 6. The resolution of each next level has consistently increased by a factor of 2, while the number of tiles - by a factor of 4. Thus, we have one tile of level 0 (0_0_0.jpg) with a resolution of 512 x 512, containing the image of the entire face; 4 tiles of level 1 (1_0_0.jpg ... 1_1_1.jpg), each containing a quarter of the image of the entire face, with a total resolution of 1024 x 1024, thus the resolution of each tile is again 512 x 512; 16 tiles of level 2, etc. For the most detailed level 6, it will be 64 * 64 = 4096 tiles, witha total resolution of 32768 x 32768. The second and third numbers in the tile's filename is the vertical index (v) and the horizontal index (u) respectively. The indexes grow from left to right and top to bottom. Here is an example of the first three levels organization:
level 0
level 1
level 2
|
| |
| |
| SpaceEngineer | Date: Wednesday, 21.12.2011, 15:52 | Message # 2 |
|
Author of Space Engine
Group: Administrators
Russian Federation
Messages: 4800
Status: Offline
| The CubeMap tool
Converting a cylindrical texture to a cubemap is performed in several stages. There is a possibility to transform only certain faces. This is controlled by the following parameters in the config file (setting the parameter to false disables all conversion phases for the corresponding face):
Create_NEG_X true
Create_POS_X true
Create_NEG_Y false # this face disabled
Create_POS_Y false # this face disabled
Create_NEG_Z true
Create_POS_Z true
Execution of each stage can be disabled by setting the corresponding parameter in the config to false or 0 (see below).
CreateBaseTex true # create a basic texture
DoConvert true # convert the selected faces
DoDownSize true # get the LOD-s selected faces
DoUpdateEdges true # update the boundaries of faces
DoTile true # get the tiles from the time raw-files
DeleteTempRAW false # remove the temporary raw-files
Consider all the steps in order.
1) Create a basic cylindrical texture
Set the parameter CreateBaseTex true to perform this operation.
SpaceEngine uses small cylindrical textures called base textures for the planets that are far from the camera. Creating a base texture occurs by a simple reduction in the original cylindrical texture by averaging the pixels in the square blocks. BaseTexDownSize parameter indicates how many times the resolution of original texture is reduced, and the BaseTexFormat also controls the format of the base texture.
Base texture must have a dimensions of 512 x 256. Do not use bigger resolution (this is have no purpose!) or non-power-of-two resolution. You may calculate the BaseTexDownSize as width of original texture divided by 512. For example, if original image is 16384 x 8192, then BaseTexDownSize = 16384 / 512 = 32. This generates a base texture of resolution 512 x 256. But if your original image is not power of two, for example 21600 x 10800, setting BaseTexDownSize = 21600 / 512 = 42 (rounded 42.1875) generates a base texture of resolution 514 x 257. So you must open it in any image editor and reduce to 512 x 256. For better precision, you may want to use BaseTexDownSize 21 (twice lower than calculated), that generates the texture with resolution of 1028 x 514, then reduce it to 512 x 256.
2) Creating cubemap texture faces from the cylindrical texture
Set the parameter DoConvert true to perform this operation.
The transformation is performed by bilinear filtering: to get the color for the current pixel on the face of the cube, floating-point coordinates are calculated for the corresponding point on the cylindrical texture, and color is obtained by linear interpolation of the four pixels, closest to this point. This method gives very good results, artifacts may be only noticeable very close to the poles. Setting the DoublePrecision true modifies the algorithm slightly: the resulting color is obtained by averaging the four values, obtained as described above, but using twice the resolution of the grid. The differences are not noticeable.
The program uses a cache memory for storing parts of the original cylindrical texture. It is therefore possible to complete the transformation of the texture without loading it into memory. This is true for large textures, such as NASA Earth texture "Blue Marble Next Generation", witch has a size of 86400 x 43200 x 3 = 10.4 GiB. The cache size is specified by MaxMem parameter (in MiB). In the current version (1.03) setting MaxMem greater than 512 MiB can reduce perfomance due to long searches in the cache. The best value is between 64-512 MiB.
The cache may be of two types - Line and Quad: it is a choice between using parameters MethodSide or MethodPolar, for the side faces (neg_x, neg_z, pos_x, pos_z) and polar faces (neg_y, pos_y) respectively. The possible values are Line and Quad (without the quotes). The Line mode produces a sampling from the input texture by short rows; the Quad mode samples the input texture by small square zones. In theory, Quad mode should be more effective for the polar faces, but in the current version (1.03) the difference is not noticeable for the majority of cases, or even vice versa, Quad cache is slower than Line, because accessing the disk cache is more frequent.
In our example, the conversion function creates a temporary raw file named level6.raw in a corresponding folder, with a resolution of 32768 x 32768.
If the AddExtraData parameter is set to true, then pixels from neighboring tiles are added to the perimeter of each tile. Thus, if the TileWidth is 512, it will give a resolution of 514 x 514 for this mode, and the resolution of the temporary file level*.raw will be increased at 2 * cubeFaceWidth / TileWidth pixels, where cubeFaceWidth is the original resolution of the file level*.raw. In our example, it will be 32768 + 128 = 32896.
Additional pixels make it easy to eliminate gaps in the landscape and sharp borders at the junction of textures when rendered using trilinear or anisotropic filtering. It is important to save tiles in lossless format for elevation map texture in this case. It is best to use 16-bit png.
The elevation maps (InputChannels 1) can be normalized to utilize the full dynamic range of integers. For example, when using a 16-bit height map "gebco_bathy.21601x10801.bin", the minimum and maximum pixel values are -10577 and +8430, that does not completely fit the dynamic range of a 16-bit integer (-32768 ... +32767). Setting the Normalize parameter to true fits the dynamic range, i.e. stretches pixels values so minimum value becomes -32768 and maximum value becomes +32767. Moreover, if the parameter EnterNormData is false, the minimum and maximum values are computed by viewing all pixels in the input texture, but it can take a long time for large textures. So it is best to perform computing of min/max values once, and then set EnterNormData to true and enter the data, which the program had previously outputted to the console:
Normalize true # normalize the output image (height map)
EnterNormData true # to enter data manually for the normalization
NormMinValue -10577 # the minimum value
NormMaxValue 8430 # the maximum value
3) Creating LODs down to zero LOD
Set the parameter DoDownSize true to perform this operation.
The temporary file resolution reduces itself twice until the tile resolution is reached. For example, a new temporary file appears in the corresponding face folder: level5.raw, level4.raw, level3.raw, level2.raw, level1.raw and level0.raw, with resolutions from 16384 x 16384 to 512 x 512.
If AddExtraData is set to true, the resolution of temporary files level*.raw will increased on 2 * cubeFaceWidth / TileWidth pixels, where cubeFaceWidth is original resolution of the file level*.raw.
4) Update the boundaries of faces
Set the parameter DoUpdateEdges true to perform this operation. This works only if AddExtraData is true.
AddExtraData mode, the pixels must be taken from neighboring faces to obtain correct values for additional pixels in tiles that are on a face's edge. Temporary files of all faces must be calculated for this, i.e. they must be completed with DoConvert and DoDownSize stages for all faces, otherwise the program will crash.
5) Creation of the tiles from the temporary raw files
Set the parameter DoTile true to perform this operation.
Temporary files are cut to the appropriate number of tiles with a resolution of TileWidth x TileWidth (or TileWidth + 2 in AddExtraData mode) and stored in the corresponding face's folder as images in one of the following formats:
raw - 8 or 16 bits, any number of channels
tga - 8 bits, 1, 3, 4 channels, uncompressed
dds - 8 bits, 1, 3, 4 channels, uncompressed
jpg - 8 bits, 1 or 3 channels, lossy compression
tif - 8 or 16 bit, 1, 3, 4 channels, lossy or lossless compression
png - 8 or 16 bit, 1, 3, 4 channels, lossless compression
The tile format is controlled by the following parameters:
OutFormat - output format (raw, tga, dds, jpg, tif, png)
TileWidth - resolution of the tile
Out16bit - bit-depth: 16 or 8 bits per channel (only for raw and png)
OutByteSwap - to 16-bit: little endian (MAC format)
OutUnsigned - to 16-bit: unsigned or signed values
OutInvertAlpha - invert the alpha channel for RGBA images
OutJPEGquality - the quality of JPEG compression (0 ... 100)
If files already exist in the folder, they will NOT be overwritten.
6) Automatically deleting of temporary files
Set the parameter DeleteTempRAW true to perform this operation.
|
| |
| |
| SpaceEngineer | Date: Wednesday, 21.12.2011, 15:52 | Message # 3 |
|
Author of Space Engine
Group: Administrators
Russian Federation
Messages: 4800
Status: Offline
| The Optimizer tool
The SpaceEngine program supports the tiled cubemap texture of any level of detail at any place on a planet. This means that you can have for example the Earth texture at a resolution of 6 x 32768 x 32768 (obtained from the NASA "Blue Marble Next Generation" texure), but in some places, like in cities , the level of detail can be increased by adding tiles of 7, 8 or more levels, or vice versa, in vast homogeneous areas with no details, such as oceans, you can remove the tiles up to 2-3 level, thus saving disk space and increase loading speed. The Optimizer tool tries to identify these homogeneous regions and remove them.
Determining homogeneity is performed by calculating the variation of color of the pixels of the tile - calculating a weighted sum of modulus of difference between current pixel color and average tile color. If this variation is less than the value of MinVar parameter from the config, the tile is considered to be homogeneous and moved to a temporary folder.
The temporary folders are created in a folder called OutFolder and has the names of del_neg_x, del_neg_y, del_neg_z, del_pos_x, del_pos_y, and del_pos_z. The homogeneous tiles are moved to these folders from the folders of neg_x, neg_y, neg_z, pos_x, pos_y and pos_z respectively. These temporary folders with their contents may be subsequently deleted.
General guidelines for working with the utility are as follows: Start with some MinVar value and run Optimizer for one face, choosing that face by set one of parameters Create_NEG_X ... Create_POS_Z to 1 or true. After that, open the temporary folders and look at the tiles that were moved by Optimizer, then correct MinVar, then move the tiles back (manually) and repeat this process several times until a satisfactory result is obtained. Then run Optimizer for the other faces with the best MinVar value found.
The current version (1.03) does not use a very good algorithm, so there may be mistakes - such as a tile with a small island and an ocean around it uniformly may be considered to be entirely homogeneous, while the tile with a slight color gradient may be considered to be inhomogeneous, although this is should be the vise versa. Therefore, it is better to view all the tiles in the temporary folder and move some tiles back that were incorrectly moved. Also, for large textures it is recommended to have a temporary folder saved somewhere safe, to quickly restore incorrectly deleted tiles without having to start the CubeMap tool again.
The Glue tool
The Glue tool is designed for bonding tiles back into a single raw-file. Only 8-bit images are supported in the current version (1.03). These are the following parameters used:
InputFile - output image file
OutFolder - the folder in which the tiles are searched
InputWidth - width of output image
InputHeight - height of output image
InputChannels - the number of channels (Grayscale - 1, RGB - 3, RGBA - 4, etc.)
Please note that InputFile and OutFolder change roles: now the first one becomes the output file, and the second one becomes the input file. In addition to this change, other options can be used, as follows:
TilePrefix - string, the prefix before the name of tile (level number)
StartU - int, the starting index of U
StartV - int, the starting index of V
SwapUV - boolean, swap U and V indices
The tile's file name is constructed as follows: TilePrefix_v_u.*, for example, if TilePrefix is 3_, StartU is 2, and StartV is 1, then the tiles will have names 3_1_2.*, 3_1_3.*, etc. Extension .* indicates that the program will find any of the supported formats itself, in the following priority: .dds, .jpg, .jpeg, .tif, .tiff, .png, .tga, .raw. I.e., for example, if a folder has two files 3_1_2.jpg and 3_1_2.tga, then the preference is given to the first.
General Recommendations
1) Placing the input and output textures on different physical disks can significantly increase perfomance. Also, PC performance increases when the input file and temporary raw-files are defragmented.
2) It is good idea to give a decent name to the file that describes its format, for example: "Earth-surface-32k-RGBA.raw" means RGBA texture with 32768 x 16384 resolution, "Mars-bump-16k-16bit.raw" means 16-bit grayscale texture with 16384 x 8192 resolution. This will help in identification and avoid confusion.
3) Converting the texture of any format to raw can be done in Photoshop, IrfanView and other such programs. They also allow viewing of these temporary raw-files.
4) Save the config files for each of the transformed textures, as they might be useful in the future. Associate files with the extension .ccf with program CubeMap.exe, then double clicking on the .ccf file will automatically start the CubeMap tool. The .ccf file can also be run by "drag and drop" onto the shortcut of the CubeMap, Optimizer or Glue executable.
5) Do not delete temporary raw-files immediately (by setting DeleteTempRAW true), they may be useful to keep, they can be removed manually at anytime.
6) If the CubeMap program should become interrupted for any reason, then it can continue on from its last position so hitting Ctrl+C may be used to stop program. For example, if the values for the parameters InputByteSwap and InputUnsigned are not known, they can be found by transforming one face and looking at the result (tiles). But if the texture is very large, then such test will take a lot of time. Therefore, you may interrupt the conversion process to see the finished part of a temporary file, or set DoConvert to false and run Cubemap again, to generate several tiles.
7) It is recommended to view the results of the work of CubeMap on its polar regions, and if there are significant artifacts present, try to set DoublePrecision to true or edit the polar faces raw files in the Photoshop. To do this, perform DoConvert for polar faces, edit raw files, disable DoConvert and perform the other remaining operations.
8) To avoid loss of accuracy, it is not recommended to normalize the elevation map with dynamic range which is "almost implemented", for example, (1 ... 252) for 8-bit or (-32000 ... 32000) for 16-bit.
|
| |
| |
| Freak2121 | Date: Thursday, 07.06.2012, 05:00 | Message # 4 |
 Space Pilot
Group: Users
Canada
Messages: 89
Status: Offline
| I've been trying to replace Jupiter's current map with a .bmp of this:
https://dl.dropbox.com/u/32146097/jupiter_css.png
But after trying to use the cubemap tool for nearly an hour with every result coming out something like this:
https://dl.dropbox.com/u/32146097/0_0_0.png
or just pure noise, I give up.
SpaceEngineer, can you tell me where I went wrong?
This is my ccf:
https://dl.dropbox.com/u/32146097/default.ccf
Edit: Got a little ahead, but the files still seem cut off and have distorted colors.
Edit: Never mind, I fixed it. It seems I overlooked the fact that I was saving my .raw as an 8-bit file but specifying it as 16-bit in the config.
Fantastic tutorial, all my messing ups were my own fault. Have everything working now.
Intel Core i5 @ 4.534GHz
8GBs of DDR3 RAM @ 1600mhz
EVGA GTX970 SSC
Windows 7 64-bit
Edited by Freak2121 - Thursday, 07.06.2012, 05:29 |
| |
| |
| Solaris | Date: Sunday, 23.09.2012, 17:43 | Message # 5 |
 World Builder
Group: Global Moderators
France
Messages: 731
Status: Offline
| Maybe one day in SE (I hope !) :
http://www.lpi.usra.edu/nlsi/moonVideo/
I've no idea how hard it would be to make such a map, but it look great.
|
| |
| |
| miros_0571 | Date: Sunday, 23.09.2012, 19:05 | Message # 6 |
 Astronaut
Group: Banned
Ukraine
Messages: 54
Status: Offline
|
Here's Addon Greyscale Moon: http://en.spaceengine.org/forum/17-906-1
MS Windows 7 Ultimate 32-bit SP1
Intel Pentium 4 CPU 3.20GHz, 2,0GB RAM, NVIDIA GeForce GTX 260
|
| |
| |
| HarbingerDawn | Date: Sunday, 23.09.2012, 19:11 | Message # 7 |
 Cosmic Curator
Group: Administrators
United States
Messages: 8717
Status: Offline
| Quote (miros_0571) Here's Addon Greyscale Moon
He didn't mean a grayscale Moon, he meant a very high resolution map.
All forum users, please read this!
My SE mods and addons
Phenom II X6 1090T 3.2 GHz, 16 GB DDR3 RAM, GTX 970 3584 MB VRAM
|
| |
| |
| Solaris | Date: Sunday, 23.09.2012, 19:15 | Message # 8 |
|
World Builder
Group: Global Moderators
France
Messages: 731
Status: Offline
| miros_0571, I was pointing the level of details of the textures/bump map used in this video.
And thanks for the link, but this cool addon is already installed in my SE 
|
| |
| |
| SpaceEngineer | Date: Sunday, 23.09.2012, 19:47 | Message # 9 |
|
Author of Space Engine
Group: Administrators
Russian Federation
Messages: 4800
Status: Offline
| HarbingerDawn, works on the hi-res Moon texture. And hi-res Moon bump map may be made from fresh LRO data. My last update was a year ago.
|
| |
| |
| Silvermistshadow | Date: Sunday, 16.12.2012, 22:10 | Message # 10 |
 Observer
Group: Newbies
United States
Messages: 6
Status: Offline
| I made a map for a few of my planets in Fractal Terrains (a demo version, since the program itself is incredibly expensive- or it was last time I looked), but it has neither raw format nor a cubemap projection available. Guess I'll have to settle for procedural generation.
|
| |
| |
| CaptainKeanu | Date: Saturday, 22.12.2012, 18:33 | Message # 11 |
 Observer
Group: Newbies
Indonesia
Messages: 2
Status: Offline
| can somebody make me a texture of a planet (i already have one) but idk how to insert to the game, pls help me and this is the texture :
|
| |
| |
| smjjames | Date: Saturday, 22.12.2012, 18:41 | Message # 12 |
|
World Builder
Group: Users
United States
Messages: 913
Status: Offline
| Uh, did you even look at the stickies?
|
| |
| |
| CaptainKeanu | Date: Saturday, 22.12.2012, 18:43 | Message # 13 |
|
Observer
Group: Newbies
Indonesia
Messages: 2
Status: Offline
| nope
|
| |
| |
| apenpaap | Date: Saturday, 22.12.2012, 19:06 | Message # 14 |
 World Builder
Group: Users
Antarctica
Messages: 1063
Status: Offline
| (smjjames) Uh, did you even look at the stickies?
I occasionally stream at http://www.twitch.tv/magistermystax. Sometimes SE, sometimes other games.
|
| |
| |
| HarbingerDawn | Date: Sunday, 23.12.2012, 02:31 | Message # 15 |
|
Cosmic Curator
Group: Administrators
United States
Messages: 8717
Status: Offline
| Hello CaptainKeanu, and welcome to the forum. Please take a moment to read the forum rules.
An appropriate place to make your post would have been here. As the others have said, you should at least try to look for the answer to your question - or an appropriate place to post it - before making a new thread.
All forum users, please read this!
My SE mods and addons
Phenom II X6 1090T 3.2 GHz, 16 GB DDR3 RAM, GTX 970 3584 MB VRAM
|
| |
| |
ENG
New site
Russian Federation 
Canada 
France 
Ukraine 
United States 
Indonesia 
Antarctica