[TOOL] X4 Customizer 1.23

The place to discuss scripting and game modifications for X4: Foundations.

Moderators: Moderators for English X Forum, Scripting / Modding Moderators

ShadowTek49
Posts: 17
Joined: Sat, 8. Dec 18, 01:06

Re: [TOOL] X4 Customizer 0.10 (semi-useful)

Post by ShadowTek49 » Thu, 13. Dec 18, 03:36

OK, let me ask this, what would everything under:

# Fill in transforms below.

look like using these paths
D:\X4\UnPack
D:\X4\Pack

for example this:
# Write modified files.
Write_To_Extension()

do I delete that? Change it?

I get:
skipped Cat_Unpack due to a Exception exception

Thats exactly what it says weather I delete that part or not

EDIT:

This is what I did

Code: Select all

# Fill in transforms below.

from Plugins import *
Cat_Unpack(
    source_cat_path = r'D:\X4\Pack\ext_01.cat',
    dest_dir_path   = r'D:\X4\UnPack'
    )
    
Thats everything below:
# Fill in transforms below.

should there be something else?

Edit 2:
Keeping:

# Write modified files.
Write_To_Extension()

gives same error msg.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 0.10 (semi-useful)

Post by SirNukes » Thu, 13. Dec 18, 03:54

For the input script, the code I posted is all that's needed for just unpacking a catalog; the rest can be commented out.

For the exception, it sounds like the md5 hash check is failing, given the place it happens. (Catalog files include hashes of each game file, and this tool computes and verifies the hash when pulling data out of the .dat file.) I'll take a look into it; in the worst case I can change it to allow md5 mismatches.

ShadowTek49
Posts: 17
Joined: Sat, 8. Dec 18, 01:06

Re: [TOOL] X4 Customizer 0.10 (semi-useful)

Post by ShadowTek49 » Thu, 13. Dec 18, 03:59

The file I'm trying to unpack is a mod not a game file if that matters.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 0.10 (semi-useful)

Post by SirNukes » Thu, 13. Dec 18, 04:48

ShadowTek49 wrote:
Thu, 13. Dec 18, 03:59
The file I'm trying to unpack is a mod not a game file if that matters.
It might, if the catalog was created with a bad hash.

At any rate, 0.10.1 is up on github with three changes:
A) It turns out the egosoft catalogs have a small bug with empty files, giving an md5 hash of 0 (when the proper empty hash is 'd41d8....'), so I added a special check for that case.
B) There is a new setting called "allow_cat_md5_errors" which will suppress the exception on a failed md5 check.
C) Cat_Unpack has an extra parameter when unpacking extension directories, to help it find the right files automatically. You can still feed it the specific cat file as before, or use the directory style example below.

So the code would now look like:

Code: Select all

from Plugins import *
Settings.allow_cat_md5_errors = True
Cat_Unpack(
    source_cat_path = r'D:\X4\Pack',
    dest_dir_path   = r'D:\X4\UnPack',
    is_extension    = True
    )

ShadowTek49
Posts: 17
Joined: Sat, 8. Dec 18, 01:06

Re: [TOOL] X4 Customizer 0.10 (semi-useful)

Post by ShadowTek49 » Thu, 13. Dec 18, 06:09

SirNukes wrote:
Thu, 13. Dec 18, 04:48
ShadowTek49 wrote:
Thu, 13. Dec 18, 03:59
The file I'm trying to unpack is a mod not a game file if that matters.
It might, if the catalog was created with a bad hash.

At any rate, 0.10.1 is up on github with three changes:
A) It turns out the egosoft catalogs have a small bug with empty files, giving an md5 hash of 0 (when the proper empty hash is 'd41d8....'), so I added a special check for that case.
B) There is a new setting called "allow_cat_md5_errors" which will suppress the exception on a failed md5 check.
C) Cat_Unpack has an extra parameter when unpacking extension directories, to help it find the right files automatically. You can still feed it the specific cat file as before, or use the directory style example below.

So the code would now look like:

Code: Select all

from Plugins import *
Settings.allow_cat_md5_errors = True
Cat_Unpack(
    source_cat_path = r'D:\X4\Pack',
    dest_dir_path   = r'D:\X4\UnPack',
    is_extension    = True
    )
OK Thanks man for your responses in helping me, gotta goto work in a few hours (damn this game lol) so I'll try it after work tomorrow, again thanks for the support :)

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 0.11 (semi-useful)

Post by SirNukes » Sun, 16. Dec 18, 09:04

Updated to 0.11. This adds command line arg support to called scripts, and uses it to make the Cat unpacker and packer more command line friendly. Note: the default script has been swapped from User_Transforms.py to Default_Script.py.

The Check_Extension utility is added, which will attempt to apply extension diff patches and report errors. This does two passes: one with the extension loaded early (after its dependencies), the other late (in case a prior extension removes or moves nodes). Check_All_Extensions will call Check_Extension on all enabled extensions, and the Check_Extensions script offers a command line method of calling the plugin. If you try it out, please let me know how well it works.

Does anyone have ideas for other utilities that would be handy?

Edit: 0.11.1 adds some support for case insensitive file paths (so extensions don't have to match the catalogs).

R_Romach_R
Posts: 48
Joined: Fri, 23. Feb 18, 16:38
x4

Re: [TOOL] X4 Customizer

Post by R_Romach_R » Sun, 16. Dec 18, 13:29

SirNukes wrote:
Sun, 9. Dec 18, 10:58
Diggz wrote:
Sat, 8. Dec 18, 23:22
I helped someone on discord do some troubleshooting. Our findings is the X4 root folder only seems to cause problems for windows 7 PCs.
Odd; I am on Windows 7 and haven't had trouble with the X4 folder extensions. Maybe it is related to file permissions.

But anyway, the customizer has gotten a lot of love over the last 30 hours or so, and the framework is mostly complete. The diff patch support needs more polish and stress testing, since I just ended up writing my own (RFC 5261 doesn't seem super popular). Customizer settings were overhauled, including the option to write to documents/egosoft.
I have problems, but putting files into documents fixes them

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 1.0 (added weapon mods)

Post by SirNukes » Wed, 19. Dec 18, 03:04

Version 1.0 has been posted. This version adds a variety of weapon transforms, and fleshes out the generated weapon tables.

New transforms: Adjust_Weapon_Damage, Adjust_Weapon_Fire_Rate, Adjust_Weapon_Range, and Adjust_Weapon_Shot_Speed. Let me know if they cause any oddities, particularly with missiles (which I haven't tested much).

The output of Print_Weapon_Stats is now in csv and html form, and includes a variety of weapon properties. Rate of fire and DPS will disagree with the in-game (1.32) encyclopedia for burst fire weapons, due to a bug in the latter.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 1.1

Post by SirNukes » Thu, 20. Dec 18, 00:45

Updated to 1.1.1.

This adds a couple ware transforms: Adjust_Ware_Prices and Adjust_Ware_Price_Spread. The latter can be used to adjust trading profit. Print_Ware_Stats will document all found wares.

Adjust_Mission_Rewards will tune the credit and notoriety rewards for generic missions by a flat multiplier.

This update also refines the generated diff patches, mainly to get around an lxml performance issue with index based xpaths which were causing slowdown when verifying large files (eg. wares.xml). As a bonus, the paths are more pleasant to look at now.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 1.2 (added gui)

Post by SirNukes » Sun, 23. Dec 18, 11:37

Updated to 1.2.

I am proud to release a GUI, as a Christmas present for anyone using this tool. It took about 4 days to put together and polish well enough to be worth releasing (while learning gui design using Qt along the way). The file size is a little bloated by the libraries used, but I think it's worth it. If you are running from the Python source code, it requires the PyQt5 package and is launched using "Framework/Main.py -gui".

The current gui features include: python syntax and plugin highlighting for input scripts, documentation viewers, a settings editor, script launching, a preliminary weapon info viewer; plus niceties like changing font, remembering layout, and processing on a background thread.

Upcoming development will be aimed at fleshing out the game info viewers: wares, ships, jobs, etc. The current weapons info view is a bit of a test on how to lay out the tab, and doesn't include all weapons. Future work could potentially add editing features to these viewers; that wouldn't be too hard to do in some cases, though needs a little thought on how to organize things.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 1.3 (added editable weapon table)

Post by SirNukes » Wed, 26. Dec 18, 06:12

Updated to 1.3

This adds framework support for editing game data by hand through the GUI, which took a couple days to set up (plus some extra due to Qt headaches). It is sorta like X3 Editor, but a bit different due to complexities that arise from a rats nest of diff patches and files cross referencing each other. I am pretty happy with how it has turned out, but please give feedback.

Weapons are set up for editing, with some limitations. Other tables are pending development, though should go relatively fast when I get to them.

To apply any hand edits to the actual game, use the Apply_Live_Editor_Patches transform in the control script. (For now, this should precede any other transforms that might read the changed values, though I have some ideas on how to let some transforms run earlier.)

An example work flow looks something like this (assuming paths in settings have been set up):
1) Open gui.
2) Go to the weapons tab and hit Refresh.
3) Play around with making some edits to the 'edited' column.
4) Go to the script tab and set up the script with Apply_Live_Editor_Patches() (can drag it over from the left pane).
5) Hit the Run Script button.
6) Go back to the weapons tab and verify the changes showed up the 'post script' column.

This release cleans up a bug when running the exe from outside the X4_Customizer folder. Also, the GUI is now the standard launch format, so the exe should be safe to double click or be a shortcut target.

lyfesgrafiq
Posts: 13
Joined: Thu, 6. Dec 18, 13:28
x4

Re: [TOOL] X4 Customizer 1.3 (added editable weapon table)

Post by lyfesgrafiq » Thu, 27. Dec 18, 11:37

Hello... I was messing around with the gui yesterday... nice job. I'm having issues with the path... and I was wondering if you could tell me what you refer to as the "control script"? Is that the default_script? I assumed it was but I could be wrong.

I had some issues but I'm trying to make sure it's not just my own errors before I say anything.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 1.3 (added editable weapon table)

Post by SirNukes » Thu, 27. Dec 18, 19:40

The documentation needs an update, but in short: the "control script" is the python script that calls all of the desired transforms (or other plugins), generally acting as a build script which constructs the custom extension. The "default_script" is a pre-gui concept, being run when the tool is run in non-gui mode but is not given a different script name. It was mostly put in for the convenience of double clicking a bat file.

But now with the gui in place, it will remember the path to the last loaded script, so the name used doesn't particularly matter.

lyfesgrafiq
Posts: 13
Joined: Thu, 6. Dec 18, 13:28
x4

Re: [TOOL] X4 Customizer 1.3 (added editable weapon table)

Post by lyfesgrafiq » Fri, 28. Dec 18, 13:19

Ok thanks for the info

Perry-Rhodan49
Posts: 11
Joined: Tue, 4. Dec 18, 16:30
x4

Re: [TOOL] X4 Customizer 1.3 (added editable weapon table)

Post by Perry-Rhodan49 » Sat, 29. Dec 18, 07:52

Sounds like a great tool @SirNukes, I guess I missed something but I can't open the zip file. I downloaded the compressed version to run it in Windows.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 1.3 (added editable weapon table)

Post by SirNukes » Sat, 29. Dec 18, 10:02

I was trying out compression with LZMA, without realizing you need 7-zip to unpack it. From a quick check just now, deflate will only be 25% larger and windows should handle it fine; I will use that in the next release (probably sometime tomorrow).

Perry-Rhodan49
Posts: 11
Joined: Tue, 4. Dec 18, 16:30
x4

Re: [TOOL] X4 Customizer 1.3 (added editable weapon table)

Post by Perry-Rhodan49 » Sat, 29. Dec 18, 11:57

SirNukes wrote:
Sat, 29. Dec 18, 10:02
I was trying out compression with LZMA, without realizing you need 7-zip to unpack it. From a quick check just now, deflate will only be 25% larger and windows should handle it fine; I will use that in the next release (probably sometime tomorrow).
Thx, would be great, seeing forward to test out your tool

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4 Customizer 1.4

Post by SirNukes » Sun, 30. Dec 18, 07:38

Version 1.4 is up.

On the surface, it adds a Wares tab with editable fields, and the Weapons tab now supports changing a laser's bullet.

Under the surface, major refinements have been made to the gui code (for easy tab expansion, thread chaining, etc.) and the live editor (lots of polish and debug). Parsing the wares.xml file was exceedingly slow with all of the xpaths I was throwing at it (lxml was bogging things down), so various speedup code was added to get its loading time down to a couple seconds. Multithreading it was a bit of a headache due to Python awkwardness (threading is one of its big weaknesses), so hopefully that doesn't give anyone trouble.

Perry-Rhodan49
Posts: 11
Joined: Tue, 4. Dec 18, 16:30
x4

Re: [TOOL] X4 Customizer 1.4

Post by Perry-Rhodan49 » Sun, 30. Dec 18, 08:57

Thanks for all your efforts.
Unfortunately, application crashes when I try to add the path for my x4 installation (Win 10)

catarix
Posts: 11
Joined: Tue, 12. Oct 10, 21:54
x4

Re: [TOOL] X4 Customizer 1.4

Post by catarix » Sun, 30. Dec 18, 09:04

Perry-Rhodan49 wrote:
Sun, 30. Dec 18, 08:57
Thanks for all your efforts.
Unfortunately, application crashes when I try to add the path for my x4 installation (Win 10)
exactly the same for me (win 10 too)

Post Reply

Return to “X4: Foundations - Scripts and Modding”