[TOOL] Changeling - X3:TC Editor and Change Manager [v2.8.1 - Sa-07-May-2011]

[Sam L.R. Griffiths]

Hi Everyone,

I started work on a Configuration Tool for the EES Mod, and even then was starting to see the seeds of something greater. As a result of this, and some recent discussions I have decided to take the next step and develop it into something alot more capable but keeping it simple enough for most people to work with. I plan to start with my EES Configuration Tool and evolve it as time goes on.

Please check here if the main TXU links do not work.

[ external image ] Enhanced Equipment Spawner (EES) v2.6 @ TXU

[ external image ] Changeling v2.8.1 (release notes) @ TXU

IMPORTANT: Changeling requires Microsoft .NET Framework v4.

If anyone wishes to join in on the development or testing then your support would be welcomed and I will add you team list (will just need to sort out the details of CM).

As of Su-27-May-2012, the project has been published on Google Code.

Changeling Team

• Developers
  • Me
• Testers (Not Formally Active Yet)
  • gmargaro
    Nafensoriel

Acknowledgements

• Egosoft - for developing such a compulsive addictive game
  The X3:TC Mod Development community - for all their great works towards enhancing the game
  apricotslice - For the kick-up-the-rear I needed to get this off the ground
  LitCube - For the initial implementation of EES and it's predecessor CWP
  Whimsy - For their work on Exscripter
  Antick952 and Vareck Raith - For their assistance in the investigation of the "EES No Lasers Bug" (which has been fixed as of EES v2.5)
  Gazz - For his assistance in identifying the full extent of the weapon swapping issues with EES.

[Sam L.R. Griffiths]

Implemented Features

1. Simple drag-and-drop interface
2. XML based configuration files
3. Reading/Writing of xxxx.xml/xxxx.pck and xxxx-Lxx.xml/xxxx-Lxx.pck files
4. Support for importing all textual file data in the t folder and resolving of cross-references for display in the tool
5. Support for assigning in-game names/descriptions to Laser configurations.
6. Import/Export/Editing of EES Configuration
7. Import/Export of MARS v4.95 Configuration
8. Editing of per-Laser MARS settings
9. Direct processing of PCK/CAT/DAT files
10. Raw XML Viewing/Editing of MCSI/MD Scripts (No Saving of Changes yet)

Planned Feature List

1. Direct processing of ZIP/RAR/7z files
2. Editing of Game start files
3. Editing of TLasers and TBullets
4. Editing of TShips, TCockpits, Dummies.txt, Bodies.txt, Components.txt
5. Editing of TWares
6. Editing of TDocks and TFactories
7. Editing of Scripts
8. Game Start editor (perhaps without needing to write ANY hard MD/scripts code by hand - or at least keep such edits to a bare minimum)
9. Creation and Application of change packages
10. Automatic generation of change packages from existing mods to support merging
11. Automated/Semi-automated Mod conflict resolution

Requested Feature List

1. Importing of XSPs

[Sam L.R. Griffiths]

Frequently Asked Questions (FAQs)

1. What is EES?
   
   Enhanced Equipment Spawner (EES) is a script that was initially developed by LitCube and distributed by vkerinav in tandem with his SRM/Xtra Jobs scripts/mods. The concept of EES is to alter the way that the game engine equips ships with weapons, shields, missiles and upgrades. Where mods add missiles and weapons, EES can help to ensure that appropriate weapons and missiles are equipped to AI ships.
   
   EES includes a replacement Vanilla Jobs file which is configured to use EES, plus vkerinav's SRM and Xtra Jobs files are already EES compliant.
   
   Currently, EES requires other mods/scripts to be altered to be EES compatable but I am endeavouring to find alternate options to allow anyone that wants to fully benefit from EES with their favourite mods to be able to do so without having to manually tweak the mod content itself.
   
   As of version 2.2 of EES, a modified OBS script was included so that any mission director script that uses the OBS will benefit without modification.
   
   On the roadmap for Changeling is support for auto-modification of script/mission director content to call EES/CWP/... in lieu of the stock equipment selection/spawning commands.
   
   DON'T PANIC: New game load times can take a minute or longer with EES installed. This is probably due to the fact that individual Jobs take slightly longer to create with EES and alot of jobs are created en mass at the time a new game is created. This does not appear to have any issues during main play or loading of existing games.
2. I get message ... everytime I load a game with EES installed, how do I fix it?
   
   The probable cause for the message is an out of date EES configuration but the "quick fix" resolution is simple.
   1. Download and install Changeling (if not already done so)
   2. In the Changeling tool, Import your game data
   3. Import your installed EES Configuration
   4. Export the EES Configuration over-writing the one you loaded
   5. Reload the game, and the message should be gone.
3. What is Changeling?
   
   Changeling is a creative content editor and change manager for X3TC.
4. Why is it called Changeling?
   
   Changeling is named after the creature from Western European folklore. In the folklore, a Changeling is the off-spring a legendary creature that has been secretly left in place of a human child. I felt the analogy of mods with the folklore Changeling was rather apt.
5. What does Changeling do?
   
   Currently, this tool's primary purpose is the generation of EES and MARS configurations. But it will soon also be able to collate mod/game content and allow the user to generate a consolidated CAT/DAT pair. Eventually, the mod will be expanded to support the creation/editing of Change Packages and the generation of Change Packages from existing mod content.
   
   If things go as I have envisioned then this will be a very powerful and easy to use tool that will facilitate the creation and merging of mod content and will even provide support for conflict resolution in terms that the end user will hopefully understand. Also the intention is to allow modders to change content that otherwise would be unthinkable (want to add translated starts for instance - Changling will handle the mutation of the 0001-Lxx files and also allow the end-user merging of multiple start packages)
6. What is a change package?
   
   A change package is a record of what has been changed/added/deleted (e.g. if a mod just changes the hull values then only the offset/factor applied will be recorded). The storage of data in this form will hopefully allow for the easy merging of mod content.
7. How do I install Changeling?
   
   Follow the following simple steps:-
   1. Download the ZIP file(s) associated with Changeling
   2. Extract the ZIP file(s) to a directory (e.g. C:\Changeling)
8. I have had an error dialog pop-up, how do I report it?
   
   Follow the following simple steps:-
   1. Capture just the error window by pressing Ctrl-Alt-Prt Scr
   2. Paste the captured window into your favourite imaging application (GIMP is highly recommended)
   3. Save as a compressed file format such as JPG or PNG
   4. Upload to a reputable site such as FileFront/GameFront, ImageShack, or perhaps TXU
   5. Finally, post in this thread the link to the error dialog image plus details such as the version of Changeling you are using, your OS and OS patch level.

[apricotslice]

Acknowledgements

• apricotslice - For the kick-up-the-rear I needed to get this off the ground

Your welcome.

Actually, thats the best laugh/coughing fit I've had in a month, so thankyou !

Practical matters.

You will need to also handle Bodies.txt and Dummies.txt, as these carry ship changes as well. Eg. BSG mod modifies both of them.

Also need to look at Components.txt. I believe this is auto generated, but best be sure.

Also look at including the current X3 XSP's. Be useful to easily incorporate them into a mod.

I'm available for brainstorming and ideas, for reality checks (yours or mine) and taking ideas to a logical conclusion instead of just the limits of the original idea. (My main gripe with L3 Devnet is that it doesnt allow ideas to be taken to their conclusions. A lot of the problems in the game were good ideas, only half done. imo. Obviously your driving, but destinations can be important to discuss options of.)

I dont get much game time, but can try to get some testing time in once you get to a decent stage. I prefer to test something nearly complete rather than bit test an ongoing alpha.

Forum time I fit into whatever else I'm doing, but game time is a whole different thing. So I'm here for forum support, and will see about testing later on.

[markdeno]

Your new tool looks just like what I have been waiting for.

Perhaps some advanced functions could include a form or auto-merge where we are given the option of which models, t-files etc to keep in the new crunched CAT/DAT and thus we only ever need one fake-patch.

[Sam L.R. Griffiths]

Your new tool looks just like what I have been waiting for.

Perhaps some advanced functions could include a form or auto-merge where we are given the option of which models, t-files etc to keep in the new crunched CAT/DAT and thus we only ever need one fake-patch.

Where the TFiles are concerned, when the tool is complete you will not need to select specific TFiles, my intention is to hide as much of the underlying game data structure as possible and show only the data it self.

[Sam L.R. Griffiths]

Also look at including the current X3 XSP's. Be useful to easily incorporate them into a mod.

At least initially, I will be leaving XSPs out of the equation since Cycrow is already working in that area with his Mod Manager tool. The tool will be able to the take output files from the Mod Manager and produce a change package from it.

I'm available for brainstorming and ideas, for reality checks (yours or mine) and taking ideas to a logical conclusion instead of just the limits of the original idea. (My main gripe with L3 Devnet is that it doesnt allow ideas to be taken to their conclusions. A lot of the problems in the game were good ideas, only half done. imo. Obviously your driving, but destinations can be important to discuss options of.)

Appreciated, once we get beyond a certain point in the development I will be actively discussing further improvements but at this moment in time there is enought to keep be busy for a little while. Saying this, all suggestions and ideas are welcome and will be considered.

I dont get much game time, but can try to get some testing time in once you get to a decent stage. I prefer to test something nearly complete rather than bit test an ongoing alpha.

Forum time I fit into whatever else I'm doing, but game time is a whole different thing. So I'm here for forum support, and will see about testing later on.

Great, for the immediate term, I am looking at coder testing only... when I get to the Beta phase I will certainly welcome your help.

[HotSake]

Have you looked at the Python-based tool Wrye Mash for Oblivion? Among many other features, it allows users to control the load order of mods and automatically merges compatible mods when possible. It might be worth looking at, if only for inspiration, when you/the team work on merge support.

[Sam L.R. Griffiths]

Have you looked at the Python-based tool Wrye Mash for Oblivion? Among many other features, it allows users to control the load order of mods and automatically merges compatible mods when possible. It might be worth looking at, if only for inspiration, when you/the team work on merge support.

Thanks for the suggestion, but if things go as I plan ordering concerns will hopefully go away. The nature of my idea for "Change Packages" should make such concerns either automatic or non-existent. I can not go into details yet because I am still bottoming out the concept for my own peace of mind. Rest assured, when I have the concept solidified I will shout it out here

[Observe]

I remember apricotslice and I (among others) discussing these things 5 or 6 years ago. This was part of the reason XFP forum (now defunct) was established back then.

Unfortunately I do not have time availability to engage in your project at this time. However, I will lend my support if and when possible.

Best of luck.

[Sam L.R. Griffiths]

Howdy peeps

I am currently working on XML/PCK file loading and reference resolution for the "t" directory. I have the XML and reference resolution code worked out but I am currently trying to delve into the deep recesses of decoding PCK files.

On picking up on some information indicating that a PCK file is basically an LZMA compressed file I have downloaded the v9.20 LZMA C# source for 7-zip and have been trying to use it in anger. But currently I am getting weird out of memory and maths overflow issues. Has anyone ever used these libraries to process PCK files? If so are you aware of any quirks that need to be mitigated?

[Cycrow]

what language are you creating the program in ?

you could make use of the spk.dll file which already has the ability to control pck files, as well as cat/dat files

[Sam L.R. Griffiths]

what language are you creating the program in ?

you could make use of the spk.dll file which already has the ability to control pck files, as well as cat/dat files

In C# using Visual C# 2010 Express.

What language is the spk.dll written in and where do I get it from? Language wise, providing it has either a COM or C-style interface then it should not be an issue (the former can be processed by Visual Studio and I can use DLL management libraries for the latter).

EDIT: Scratch that question... found it (it is written in C++ for those who might be wondering)

EDIT #2: Hmmm... not much help on a direct use front for now, but it might help me resolve the issues with my existing code... thanks for the pointer

[Sinnerman264]

Is there a way we could help you? I myself have no knowledge of the programming itself but I could offer hours of testing if needed. Or if there is something else I could do?

[Sam L.R. Griffiths]

Is there a way we could help you? I myself have no knowledge of the programming itself but I could offer hours of testing if needed. Or if there is something else I could do?

Not at this point, but there will be a raft of try to break it testing soon (hopefully).

I am currently implementing the reading of the text file folder and the processing of PCK files (which should mean all text will be shown in the tool pretty much as it is seen in the game). When I have that sorted, there will be some "try to break it testing" with regards to the Laser/Missile names and descriptions (ETC Su-20-Mar-2011).

The resolution code should handle circular cross-references and multiple languages. Currently I am hardcoding the language to English (44) until I have got to the next step which will involve a test release and a language selection dialog/menu.

[Sam L.R. Griffiths]

EDIT: Scratch that question... found it (it is written in C++ for those who might be wondering)

EDIT #2: Hmmm... not much help on a direct use front for now, but it might help me resolve the issues with my existing code... thanks for the pointer

I am starting to wish I started this project in C++, trying to recode spk.dll in C# is turning out to be a nightmare (alot more work than I planned - the SPK.DLL code is not written in a cross platform safe manner as it makes assumptions about byte ordering).

I have found a C# ZLib.NET library and I presume that PCK files are pure ZLib streams?

EDIT: Ok.. PCK Files appear to be ZLib compressed files with a custom header. I tried using Zlib.NET directly on the PCK files in my t directory but it complains that it does not recognise the compression, conversly using the PCK detection algorithm does not work (the Header is not recognised by the SPK.DLL approach).

I have found a Microsoft compression library GZip/Deflate/ZLib and tried each of the decompression streams and got it to work with GZip.

Now for CAT files, so I can get at those pesky 0001-Lxxx files

[Sam L.R. Griffiths]

CAT files are going to take a while to sort out, in the mean-time please find posted Changeling v2.1 (versioning continued from EES Configuration Tool)

[Sam L.R. Griffiths]

Status Update
Courtesy of some code and a couple of DLLs from X3Editor 2 (I need to verify permissions for the DLLs x2fd.dll and x3vfs.dll), I have managed to get the tool to read the entire text library, TLasers and TMissiles.

I also have some preliminary code to read TBullets and most other TFiles, but interpreting the data is another matter.

I am currently considering adding support for import/export of MARS configurations so if anyone has information on the structure of the config file it would be helpful.

In the meantime, I am going to polish off the TBullets code enable language mode selection and look to release a version hopefully sometime later today.

[Sam L.R. Griffiths]

Status Update
Release of Changeling v2.2! This is still under development, but it may still be useful to the general community in it's current form.

Main Changes

• Use of doubleshadow's x2fd.dll and x3vfs.dll to access the game's resources.
• Loading of TLasers, TBullets, TMissiles, and all the xxxx and xxxx-Lxxx files directly (NOTE: To access CAT files Changeling may require to be run with Administrator rights)
• Support for compressed session files (*.ses) as well as raw XML session files
• Support for displaying game text resources (main UI still English only) in English/German/French/Russian/Italian/Spanish via a Language menu
• General UI Look and Feel improvements

Any and all feedback is welcomed, intending on looking into implementing initial support for Change Packages and the editing of Weapon/Missile data. Once the concept is proven there, I will then move on to the export feature before moving on to other resources such as Ships, Wares, and editing of text resources.

[HotSake]

How do you envision change packages working? An XML file with a custom extension registered to Changeling, with information like "Delete these 10 jobs" and "Add these 5 factories" with Changeling interpreting all cumulative changes to the same file and making any that don't conflict? Then prompting the user to resolve conflicts?