ENG New site

Advanced search

[ New messages · Forum rules · Members ]
  • Page 1 of 1
  • 1
Forum » SpaceEngine » Mods and Addons » SpaceEngine folders and pak files 0.980
SpaceEngine folders and pak files 0.980
SpaceEngineerDate: Sunday, 01.02.2015, 10:19 | Message # 1
Author of Space Engine
Group: Administrators
Russian Federation
Messages: 4800
Status: Offline
Last update: 05.09.2016 (version 0.9.8.0)

SpaceEngine folders

SpaceEngine has two major folders with data files, which can be found in its root (installation) directory: data and addons. They are equivalent in many respects. The first contains the standard SpaceEngine files, which you should not modify. The second mirrors the internal structure of the first one, and is used for user mods and addons. It has a higher priority when searching for files at startup. This means that if you have some file in the addons folder with the same path and name as a file in the data folder, SpaceEngine will load the file from addons. This is useful for creating mods which change some of the system files (such as shaders), without actually touching the system files.

Internally, the data folder has these standard subfolders inside: catalogs, locale, models, music, scripts, shaders, sounds, textures. 'Standard' means that SpaceEngine expects to find specific data files in these subfolders, for example, music files must be in data/music/, and star catalogs must be in data/catalogs/stars/ (some of these subfolders have their own next-level standard subfolders inside).

The addons folder may also contain these standard subfolders, which work as mirror of the subfolders in data. This means that SpaceEngine will load data files from both data and addons as if they were a single large folder (but with higher priority for addons in case of a file name conflict, as has been described above).

The addons folder also allows the use of additional internal subfolders, for better organization of addons. These additional subfolders may have any name (which obviously cannot match the name of the standard subfolder), typically representing the name of an addon or a franchise, for example addons/StarWars/. Internally, the additional subfolder may contain standard subfolders with data files: catalogs, models, textures and so on, or it may contain pak files (see below), or it may contain a next-level additional subfolder, for example addons/StarWars/EmpireShips/. SpaceEngine allows for an unlimited amount of nested subfolders (limited by OS, typically the path length must not exceed 256 characters).

The pak files

The pak files are zip archives provided for better organization and distribution of game content. For example, a large addon with thousands of files may be zipped into a single pak file and distributed in this form, which is more convenient for the user. Zip compression can also be used while creating the pak file, which is useful to save disk space and download traffic.

The pak files are zip archives that include files and folders with the same structure as they have on the disk, starting from the data or addon root level (or addon/YourCustomFolder/). This means that the uppermost folder which can be zipped into a pak file is one of the standard subfolders: catalogs, models, textures and so on. SpaceEngine reads the content of all pak files at startup and uses them as a virtual file system with date priority. This leads to many consequences:

1) The pak file format
The pak files are zip archives (i.e. .zip files with the extension renamed to .pak). You may associate the .pak file type with WinRar or 7zip or whatever program you use to work with archives, and open any .pak file and extract files from it. Creating a .pak file is easy too: just select needed files and folders and compress them as .zip, then rename it to .pak when finished.

2) Virtual path
You may think of a pak file as a container for files/folders. The internal folder structure inside the pak file is a continuation of the SE folder structure. This means that SE looks for files and folders on the disk and inside all pak files as if there were no pak files and all their content is extracted at the place where these pak files are. We will call this a virtual path. The path of the disk in this article will be colored green, and inside the pak file will be colored orange. For example, these three virtual paths are equal (remember that the StarWars folder is optional):
addons/StarWars/StarWarsPlanets.pak/catalogs/planets/Coruscant.sc (packed into pak file)
addons/StarWars/catalogs/planets/Coruscant.sc (unpacked on the disk)
addons/catalogs/planets/Coruscant.sc (unpacked addon content moved to the addons root - still works)

3) Priority by modification date
If there are two or more files with the same virtual path, SE will load the most recent one. It compares the modification date/time of the disk files, or those of the entire pak file. For example, consider that you have 3 files named SolarSys.sc in different places, but with the same virtual path:
data/catalogs/Catalogs0980.pak/planets/SolarSys.sc in the default pak file, pak date is 2016/08/06
data/catalogs/planets/SolarSys.sc unpacked on the disk, file date is 2016/09/01
data/catalogs/SolarSysUpdate001.pak/planets/SolarSys.sc in another pak, pak date is 2016/09/05
In this case, SpaceEngine will load the last one, because it has the most recent date. This is useful for making updates. Consider someone adding an addon with this file:
addons/SolarSystemMod.pak/catalogs/planets/SolarSys.sc pak date is 2016/08/15
In this case, SpaceEngine will load the file from this addon pak, despite its older date. This is because files in the addons folder have priority over files in the data folder. This is to prevent broken addons after updating the default SE files. Inside the addons folder, the same file/pak modification date priority rule works.

4) Extracting files
You may extract any file from the pak archives by opening it as a zip archive. When extracting the file, you must create all the folders contained inside the pak, so the extracted file will have the same virtual path as the packed one (like use 'extract here' option in archiver program). For example, if you want to extract the file SolarSys.sc from the Catalogs0980.pak, you must create the folder data/catalogs/planets/ and extract that file into it so that it has this path:
data/catalogs/planets/SolarSys.sc
After this, SpaceEngine will load the extracted file, because it will have the more recent date. Now you may modify it.

5) Creating pak files
For example, consider you are making a planetary system addon which has these files:
addons/StarWars/catalogs/stars/StarWarsSystems.sc
addons/StarWars/catalogs/planets/Coruscant.sc
addons/StarWars/catalogs/planets/Tatooine.sc

When you finish creating it, have tested everything, and are ready to release, go to the StarWars folder, select the catalogs folder and pack it into a zip archive. Rename that archive to StarWarsPlanets-v001.pak. The file name doesn't really matter, but it should reflect the file's content, and having a version/revision number in it is also a good idea. After that, delete the catalogs/ folder and its content (or move it to some backup place) and leave only the newly created pak file. If you did the packing correctly, your addon should work in SE as before. Your data files will have this virtual path:
addons/StarWars/StarWarsPlanets.pak/catalogs/stars/StarWarsSystems.sc
addons/StarWars/StarWarsPlanets.pak/catalogs/planets/Coruscant.sc
addons/StarWars/StarWarsPlanets.pak/catalogs/planets/Tatooine.sc
So you are ready to share the file StarWarsPlanets-v001.pak with the community. Note that it will work if it is placed in any subfolder in the addons folder (except standard subfolders of course):
addons/StarWarsPlanets-v001.pak
addons/StarWars/StarWarsPlanets-v001.pak
addons/blabla/ololo/123/StarWarsPlanets-v001.pak
You are free to put any set of data files in a single pak file. For example, you may create a full Star Wars franchise pak, which includes all planetary systems, galaxy and galaxy model, and all ship models. Note that there are some restrictions.

6) Restrictions
1) The SpaceEngine root folder cannot be used as a place for pak files. This will not work:
SpaceEngine/StarWars.pak/data/catalogs/planets/Coruscant.sc
2) The pak file cannot contain additional subfolders, only default ones are allowed. This will not work:
SpaceEngine/addons/StarWars.pak/StarWars/StarWarsPlanets/catalogs/planets/Coruscant.sc
3) The pak file should be less than 4 GiB in size. If your addon does not fit into that size, split your addon into several smaller pak files.
4) Music and sound files in ogg format cannot be zipped into pak files.

7) Compression rate
Zip archives can be created with or without compression. SpaceEngine supports only the "Deflate" compression method. Some file types have a great compression rate (text files such as scripts and catalogs), some have a medium compression rate (binary files such as ship models .sm, .dds and uncompressed .tga images), some almost cannot be compressed (.jpg, .png, .tif images). It is a good idea to save a large set of image files (such as a planet textures addon) into an archive without compression, because it has great extracting/loading performance. Higher performance is always better than saving few megabytes of disk space. You could experiment to find better options for your addon. For example, you may save textures into a pak without compression, and scripts with models into another pak file with compression.

8) Never touch the default pak files
Just like in all other games, changing the default game resources is a bad idea. In the future, the game client will check the default files for changes and will kick you from the server if it finds that the default files have been changed. So remember these simple rules:
1) If you want to add something, create your own pak file (or simply store files in the folders, non-packed).
2) If you want to modify something, extract the file from the default pak and modify the extracted file.
 
quarior14Date: Monday, 12.09.2016, 15:36 | Message # 2
World Builder
Group: Users
Pirate
Messages: 649
Status: Offline
I spotted some error :
The first :
Quote SpaceEngineer ()
Last update: 05.09.2006 (version 0.9.8.0)

I think it is rather Last update: 05.09.2016 (version 0.9.8.0), because 0.9.8.0 did not exist at the time, and also the first version published Space Engine, according to the site, it was created in 2011.

The second :
Quote SpaceEngineer ()
addons/SolarSystemMod.pak/planets/SolarSys.sc

I think you mean addons/catalogs/SolarSystemMod.pak/planets/SolarSys.sc or addons/SolarSystemMod.pak/catalogs/planets/SolarSys.sc.
And then same error for these three lines :
Quote SpaceEngineer ()
addons/StarWars/StarWarsPlanets.pak/stars/StarWarsSystems.sc
addons/StarWars/StarWarsPlanets.pak/planets/Coruscant.sc
addons/StarWars/StarWarsPlanets.pak/planets/Tatooine.sc

They should be :
addons/StarWars/catalogs/StarWarsPlanets.pak/stars/StarWarsSystems.sc
addons/StarWars/catalogs/StarWarsPlanets.pak/planets/Coruscant.sc
addons/StarWars/catalogs/StarWarsPlanets.pak/planets/Tatooine.sc
or :
addons/StarWars/StarWarsPlanets.pak/catalogs/stars/StarWarsSystems.sc
addons/StarWars/StarWarsPlanets.pak/catalogs/planets/Coruscant.sc
addons/StarWars/StarWarsPlanets.pak/catalogs/planets/Tatooine.sc





Quarior

Edited by quarior14 - Monday, 12.09.2016, 15:43
 
SpaceEngineerDate: Monday, 12.09.2016, 22:18 | Message # 3
Author of Space Engine
Group: Administrators
Russian Federation
Messages: 4800
Status: Offline
Thank you quarior14, fixed.




 
AlekDate: Tuesday, 13.09.2016, 02:31 | Message # 4
Pioneer
Group: Users
United States
Messages: 326
Status: Offline
Quote quarior14 ()
Last update: 05.09.2006


I know this is irrelevant to the thread but-- more proof SpaceEngineer is a time traveler!





Living among the stars, I find my way. I grow in strength through knowledge of the space I occupy, until I become the ruler of my own interstellar empire of sorts. Though The world was made for the day, I was made for the night, and thus, the universe itself is within my destiny.
 
quarior14Date: Tuesday, 13.09.2016, 08:40 | Message # 5
World Builder
Group: Users
Pirate
Messages: 649
Status: Offline
Quote SpaceEngineer ()
Thank you quarior14, fixed.

You're welcome.

Quote Alek ()
I know this is irrelevant to the thread but-- more proof SpaceEngineer is a time traveler!

If that were the case, he already would have the final version.





Quarior
 
MosfetDate: Tuesday, 13.09.2016, 15:11 | Message # 6
World Builder
Group: Users
Italy
Messages: 754
Status: Offline
Quote quarior14 ()
Quote Alek ()
I know this is irrelevant to the thread but-- more proof SpaceEngineer is a time traveler!

If that were the case, he already would have the final version.


EPILOGUE

-"...final version.".

Vladimir was reading the last sentence with a growing discomfort. "so, finally it comes... I should have seen it early" he thought.

An holographic projection was pulsing before his eyes. The big warning glyph was rotating on five dimensions and smelling like wet aluminum.
He sighed, starting the emergency shutdown procedure, mumbling "That, was the final version, my dear sim..."

SHUTTING DOWN.
.
.
.

REBOOT

"Let there be light!" So there was light.





"Time is illusion. Lunchtime doubly so."
Douglas N. Adams
My mods
Asus x555ub: cpu i5-6200u - ram 4gb - gpu nvidia geforce 940m 2gb vram
 
lipchak09Date: Monday, 21.11.2016, 12:55 | Message # 7
Observer
Group: Newbies
Pirate
Messages: 1
Status: Offline
Hello, a question install the "SolarSystemHD and SolaSystemUltra" where specified but not loaded to the textures, is some problem with the current version, ty.
 
MosfetDate: Monday, 21.11.2016, 14:01 | Message # 8
World Builder
Group: Users
Italy
Messages: 754
Status: Offline
Please attach your se.log file located under your SpaceEngine /system folder. If you mean that surfaces are blurry and not very detailed, it's not a bug, see the FAQ at http://en.spaceengine.org/index/faq/0-29#troubleshooting




"Time is illusion. Lunchtime doubly so."
Douglas N. Adams
My mods
Asus x555ub: cpu i5-6200u - ram 4gb - gpu nvidia geforce 940m 2gb vram
 
Forum » SpaceEngine » Mods and Addons » SpaceEngine folders and pak files 0.980
  • Page 1 of 1
  • 1
Search: