[TOOL] X4LogAnalyzer - Help analyse the trade log

The place to discuss scripting and game modifications for X4: Foundations.

Moderators: Moderators for English X Forum, Scripting / Modding Moderators

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

[TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Fri, 28. Dec 18, 03:17

DISCLAIMER
In order to be able to understand a little bit better what my ships were doing I developed a little application called X4LogAnalyzer.

The goal for this application is to extract the information from the X4 Save game log and put it in a way that is easier to understand. It in no way harm your save game as it will make a copy and then work with the copy and it also do not alter anything inside of it (so you can still venture), but nevertheless I suggest to do a backup before using the application.

SOURCE CODE
The source code can be downloaded here:
https://github.com/magictripgames/X4Mag ... ogAnalyzer

HOW TO INSTALL/USE THE APPLICATION
The application requires .Net Framework 4.6.1 and does not requires installation. To use it simple download it from :
https://drive.google.com/open?id=1S5YQ2 ... zy2979c-ZT

Unzip it and look for the file called X4LogAnalyzer.exe and run it. You will see the Operations Window:
Image

In the center of the window you have 2 buttons:
"Select Log File" - this button is used to select and import the information's from your save file. Simple click it and enter the full path of the save file you want to import and hit OK.
Image

After you hit OK your save will start to be imported. This operation takes a couple of seconds and the application will be irresponsible during it (sorry, still working on it). After the import is complete you can click on other tabs to analyze the information.

"Clear Cached Data" - When you import the log file a file called X4LogAnalyzerTempXML.json is created and it stores all the information extracted from the save file for later usage. The Clear Cached Data when clicked will delete this file so you can start a new process. Usually this is used when you started a new save and need to clean the old data.

After you imported your Save File you can now use the application to analyze the information using it's 3 main functionalities:


SHIP INFORMATION:
Image
The goal of this functionality is to show which transactions were imported for each ship/station. It's columns are:
Time: When the even occurred
Ship ID: The ID of the ship that generated the event.
Ship Name: It's the full name of the ship. If you changed the ship name during your gameplay the last name should be the one displayed here.
Product: What was the Ware that was sold by that ship
Destination: To whom the Ware was sold
Quantity: How many items were sold
Item Price: For how much each item was sold
Total: Quantity * Item Price
Estimated Profit: (Item Price - Ware Min Price) * Quantity
Ship Inventory used: Quantity * Ware Weight

You will also be able to see the Time in Service, which mean the amount of time from the first trade operation to the last, Items Traded is the total of items traded by that ship/station and Total Value Traded is the total of money received by that ship (not the profit).

SHIP ANALYSIS:
Image
This functionality is used to compare the ships/stations to check which one is bringing more money to our empire. For each ship you can add 4 different "pens":
Show Individual Sales - Off and Deduce Estimate Ware Cost - OFF: This will bring the accumulated money received over time, basically an accumulated of what is been reported on the log. This is good when you are checking miners, for example as their material cost is 0 or for Stations that produce the material.

Show Individual Sales - Off and Deduce Estimate Ware Cost - On: This is the estimated accumulated profit. This is good for traders as they only report the final sale value, but the actual profit.

Show Individual Sales - On and Deduce Estimate Ware Cost - Off: This will show how much money was received in each sale and it's useful to indicate Wares that are losing value over time (For example, to indicate that a market is saturated, like Silicon)

Show Individual Sales - On and Deduce Estimate Ware Cost - On: This will show how much profit you are making in each transaction. Again a decline here may indicate a market stagnation or a lack of supply.

To add a pen to the graph simple select which option you want above and then double click the ship. You can add multiple ships or the same ship with different visions. To clean the graph just select a different tab on top and return to this tab.

WARE ANALYSIS:
Image
This functionality is used to show the performance of each Ware. It will only show the Wares that are been traded in your empire and it have 3 views:
Show Estimated Profit: Will order the wares according to the estimated profit made by each one. Again this is useful for the Trades where you are buying from the other factions.

Show Full Money Earned: Will show how much money was reported by a given Ware.

Shot Total Items Sold: Will show how many items of each ware were sold by the empire.

If you click in one of the columns it will show which ships reported selling that specific ware. This is particularly useful to check the performance of each ship in each specific ware.

To go back to the graph just click on the respective selection box.

I appreciate any feedback and suggestions.
Last edited by magictrip on Wed, 9. Jan 19, 00:46, edited 3 times in total.

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Fri, 28. Dec 18, 03:17

FAQs:
- My game is in <INSERT LANGUAGE HERE>, can I still use this application?
A> I only tested the application with the English log and as it relies on a text based log, it does not work by default. I included 2 config files, one called Wares.json and the other Configuration.Json (this one is generated the first time you run the application) which contains the names of the wares in English and the log markers I'm using respectively. So, if you want to try you can translate the Ware names and also translate the expressions: sold, to, in and Trade Complete that are in those files. After that if the application still do not work please provide the translated files alongside a Save file in your language and I will try to "internationalize" it.

- The application runs fine but when I try to import the log nothing happens
A> The application relies on the logs generated by the Vanilla game, some mods disable those logs so if that is happening the application will not be able to load the log files. Also, if you hit the "Clean log" button inside the game the save game log will be also deleted and the application also cannot recover that. If you have already imported the log then you can hit clean log as the application will keep a history of what it already imported.

Known bugs:
Sometimes the application will fail to clean the temporary XML file created to import the saves. To solve this problem just close the application and delete the file manually.

Change Log:
- v 0.1 - Initial Realease
- v 0.2 - Fixed bug on Ware Analysis where if you load a new save file it was not keeping the cached information
- v 0.3 - The application will not show a message informing the error in case it cannot process the selected Save File
- v 0.4 - Added PARTIAL EXPERIMENTAL support for TaterTrade Mod. This support was not discussed with the Mod maker so it may break at any time and/or not be fully compatible.
- v 0.5 - Fixed issue with not taking into account translated string length.
Fixed issue where SoldTranslation config were been saved everytime
Added German reference files - Thanks to @P.Hastig
- v 0.6 - Fixed an issue with an unknown mod adding a (xxx Cr) after the ware name which was making it not possible to find the wares
Changed the wares tab behavior to now show also wares that have a negative income, this would only be shown if some mods are active (don't know which one though)
Last edited by magictrip on Fri, 11. Jan 19, 18:46, edited 6 times in total.

Perry-Rhodan49
Posts: 11
Joined: Tue, 4. Dec 18, 16:30
x4

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by Perry-Rhodan49 » Sat, 29. Dec 18, 07:58

Sounds like a great tool @magictrip. I have a german savegame file. If I run your tool, I got no output due to the missing translations I guess. So I used as suggested the Configurations.json like

[{"Key":"LastSaveGameLoaded","Value":"C:\\Users\\XY\\Documents\\Egosoft\\X4\\XXXXXXXXX\\save\\save_002.xml.gz"},{"Key":"SoldTranslation","Value":"verkaufte"},{"Key":"TradeCompletedTranslation","Value":"Auftrag abgeschlossen"},{"Key":"InTranslation","Value":"in"},{"Key":"ToTranslation","Value":"an"},{"Key":"SoldTranslation","Value":"sold"}]

but the LogAnalyzer crashes then without error message. Did I miss something?

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Sat, 29. Dec 18, 21:35

Perry-Rhodan49 wrote:
Sat, 29. Dec 18, 07:58
Sounds like a great tool @magictrip. I have a german savegame file. If I run your tool, I got no output due to the missing translations I guess. So I used as suggested the Configurations.json like

[{"Key":"LastSaveGameLoaded","Value":"C:\\Users\\XY\\Documents\\Egosoft\\X4\\XXXXXXXXX\\save\\save_002.xml.gz"},{"Key":"SoldTranslation","Value":"verkaufte"},{"Key":"TradeCompletedTranslation","Value":"Auftrag abgeschlossen"},{"Key":"InTranslation","Value":"in"},{"Key":"ToTranslation","Value":"an"},{"Key":"SoldTranslation","Value":"sold"}]

but the LogAnalyzer crashes then without error message. Did I miss something?
Well, I don't really know, maybe the Wares need to be translated as well. Could you please provide a Save game so I can check what is exactly the error that is happening?

PS: I can provide the source code as well if you want to debug it yourself

Daystarz
Posts: 1
Joined: Mon, 31. Dec 18, 15:49

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by Daystarz » Mon, 31. Dec 18, 15:51

The application just crashes for me after inputting my save file. Do i need to install the "Application Manifest"? I already have the .net framework installed.

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Mon, 31. Dec 18, 23:21

Daystarz wrote:
Mon, 31. Dec 18, 15:51
The application just crashes for me after inputting my save file. Do i need to install the "Application Manifest"? I already have the .net framework installed.
Not really, I tested on my son computer and it worked fine. Can you provide your save file so I can check what is the problem?

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Mon, 31. Dec 18, 23:38

Daystarz wrote:
Mon, 31. Dec 18, 15:51
The application just crashes for me after inputting my save file. Do i need to install the "Application Manifest"? I already have the .net framework installed.
Ok, just uploaded a new version and it now should show the error message while importing the file, you can download it here:
https://drive.google.com/open?id=1S5YQ2 ... zy2979c-ZT

THE_WESTi
Posts: 22
Joined: Sun, 31. Aug 14, 04:08
x3ap

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by THE_WESTi » Mon, 31. Dec 18, 23:54

Great tool - helped me to quickly identify which products are making the best margins. Am going to setup a profit focused trade fleet to target these products.. leaving my existing traders to trade every legal ware so that they keep the economy going a bit.

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Tue, 1. Jan 19, 00:37

THE_WESTi wrote:
Mon, 31. Dec 18, 23:54
Great tool - helped me to quickly identify which products are making the best margins. Am going to setup a profit focused trade fleet to target these products.. leaving my existing traders to trade every legal ware so that they keep the economy going a bit.
Good to know it helped you, my goal was exactly the same as you :-)

yafeshan
Posts: 19
Joined: Fri, 30. Nov 18, 15:59
x4

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by yafeshan » Tue, 1. Jan 19, 09:14

Today, I just entered forum to look for something like this. Great work my friend.

I am testing data of my existing station and want to ask about data. Is it possible distinguish a trade sell and buy. Some items from my self sufficient station show 0 profit and I am puzzled about this. I am guessing some of them can be a buy but not all

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Tue, 1. Jan 19, 09:23

yafeshan wrote:
Tue, 1. Jan 19, 09:14
Today, I just entered forum to look for something like this. Great work my friend.

I am testing data of my existing station and want to ask about data. Is it possible distinguish a trade sell and buy. Some items from my self sufficient station show 0 profit and I am puzzled about this. I am guessing some of them can be a buy but not all
I can only see what the game logs, sadly every time you sell to yourself, to your own ships or a station subordinate sells the ware they will not generate a log.

So, if your station is selling to other factions directly, that will show in the log and you will see it in the application. Also, if you set your Auto Traders manually instead of been station subordinates it will also show that ship selling the item. It's not perfect but works just fine.

Also, the game does not log how much you paid for a ware, that is why the profit is an estimation based on the min price, although a lot of times you are paying more than the min price.

alexringess
Posts: 40
Joined: Tue, 25. Sep 18, 23:57
x4

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by alexringess » Wed, 2. Jan 19, 18:50

It doesn't work with non gziped save files.

Lord Simpson
Posts: 28
Joined: Sat, 31. Jan 09, 20:20
x4

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by Lord Simpson » Sat, 5. Jan 19, 20:48

Hi
Thanks for building this! With the X games so focused on economy simulation you would think by now we would have at least some clear and useful reporting in-game!

However the analyser seams to have stopped working for me at the import stage, all was working fine then I Installed the below mod to throw in some extra log messages in but it seams to have broken the import process of the Analyser. Not sure if you could add compatibility for the below message structures and/or bypass any ones that the analyser doesn't pick up correctly (assuming this mod is the cause of the error and its not just a coincidence).

https://www.nexusmods.com/x4foundations ... escription

The logs the mod adds appear to be in the format

[Ship Name] [Ship ID] bought [amount] [ware] from [Station Name] [Station ID] in [Sector] for [Credits]

Thanks again for a great tool!

P.Hastig
Posts: 20
Joined: Thu, 14. Jun 07, 14:19
xr

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by P.Hastig » Sun, 6. Jan 19, 23:31

magictrip wrote:
Sat, 29. Dec 18, 21:35
Perry-Rhodan49 wrote:
Sat, 29. Dec 18, 07:58
Sounds like a great tool @magictrip. I have a german savegame file. If I run your tool, I got no output due to the missing translations I guess. So I used as suggested the Configurations.json like

[{"Key":"LastSaveGameLoaded","Value":"C:\\Users\\XY\\Documents\\Egosoft\\X4\\XXXXXXXXX\\save\\save_002.xml.gz"},{"Key":"SoldTranslation","Value":"verkaufte"},{"Key":"TradeCompletedTranslation","Value":"Auftrag abgeschlossen"},{"Key":"InTranslation","Value":"in"},{"Key":"ToTranslation","Value":"an"},{"Key":"SoldTranslation","Value":"sold"}]

but the LogAnalyzer crashes then without error message. Did I miss something?

Well, I don't really know, maybe the Wares need to be translated as well. Could you please provide a Save game so I can check what is exactly the error that is happening?

PS: I can provide the source code as well if you want to debug it yourself

Hi magictrip,

i also have a german savegame, so i translated both files (Configurations.json and Wares.json). Unfortunately your tool still crashes with exception:

Error found while trying to import the save game. Error description: Die Eingabezeichenfolge hat das falsche Format. (translated: the input string has the wrong format).

I tried to compile and debug your source but too many assemblies are missing (e.g. AvalonEdit, LiveCharts..).

Here're the config files and the savegame:
https://drive.google.com/drive/folders/ ... sp=sharing

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Mon, 7. Jan 19, 01:34

P.Hastig wrote:
Sun, 6. Jan 19, 23:31
Hi magictrip,

i also have a german savegame, so i translated both files (Configurations.json and Wares.json). Unfortunately your tool still crashes with exception:

Error found while trying to import the save game. Error description: Die Eingabezeichenfolge hat das falsche Format. (translated: the input string has the wrong format).

I tried to compile and debug your source but too many assemblies are missing (e.g. AvalonEdit, LiveCharts..).

Here're the config files and the savegame:
https://drive.google.com/drive/folders/ ... sp=sharing
Hello P.Hastig. I was able to find a bug that was introduced when I added support for TaterTrade. Please download the new version and it should work for you now.

Please let me know if you are still having problems.

PS: I added your files in the new version so other people can use it if needed :-)

P.Hastig
Posts: 20
Joined: Thu, 14. Jun 07, 14:19
xr

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by P.Hastig » Tue, 8. Jan 19, 20:44

magictrip wrote:
Mon, 7. Jan 19, 01:34
Hello P.Hastig. I was able to find a bug that was introduced when I added support for TaterTrade. Please download the new version and it should work for you now.

Please let me know if you are still having problems.

PS: I added your files in the new version so other people can use it if needed :-)
Thank you,

the new version now can handle german savegames, but there is still a little issue.
It seems, that product-strings with special characters like ä, ö, ü can't be parsed.
That's strange, because destination-strings like "ARG Hüllenteilefabrik I" where parsed correctly.

See here..:
Image

..and the corresponding rows in the savegame file:

Code: Select all

<entry time="147056.314" title="Handel abgeschlossen" text="Hermes Angreifer QRT-394 verkaufte 50 Sensorenbündel an ARG Argon Raumdock JCB-827 in Argon Prime für 104074 Cr." faction="{20203,201}" money="10407450"/>
<entry time="150377.804" title="Handel abgeschlossen" text="Hermes Angreifer QRT-394 verkaufte 10 Sensorenbündel an ARG Geschützturmkomponenten-Fabrik I DBX-715 in Argon Prime für 18861 Cr." faction="{20203,201}" money="1886130"/>
It's a newer savegame (save_005.xml.gz), i upoaded it to the same directory.

Also i made a little mistake translating wares.json in the first row. The char 'e' at the end was missing: "Name": "Fortschrittliche Verbundwerkstoffe".

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Tue, 8. Jan 19, 21:30

P.Hastig wrote:
Tue, 8. Jan 19, 20:44
Thank you,

the new version now can handle german savegames, but there is still a little issue.
It seems, that product-strings with special characters like ä, ö, ü can't be parsed.
That's strange, because destination-strings like "ARG Hüllenteilefabrik I" where parsed correctly.

See here..:
Image

..and the corresponding rows in the savegame file:

Code: Select all

<entry time="147056.314" title="Handel abgeschlossen" text="Hermes Angreifer QRT-394 verkaufte 50 Sensorenbündel an ARG Argon Raumdock JCB-827 in Argon Prime für 104074 Cr." faction="{20203,201}" money="10407450"/>
<entry time="150377.804" title="Handel abgeschlossen" text="Hermes Angreifer QRT-394 verkaufte 10 Sensorenbündel an ARG Geschützturmkomponenten-Fabrik I DBX-715 in Argon Prime für 18861 Cr." faction="{20203,201}" money="1886130"/>
It's a newer savegame (save_005.xml.gz), i upoaded it to the same directory.

Also i made a little mistake translating wares.json in the first row. The char 'e' at the end was missing: "Name": "Fortschrittliche Verbundwerkstoffe".
Well, I believe the difference there is that for ship names I use the identifier (like QRT-394) to do the queries and searches, while the wares I use the name, I may have to remove special characters or something like that for them to behave correctly. So, I will need to check that.

PS: Thanks for the fixed wares.json, I will upload to the new version as well.

User avatar
nonnex
Posts: 366
Joined: Wed, 28. Aug 13, 12:42
x4

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by nonnex » Tue, 8. Jan 19, 22:02

convert haystack and needle strings to to UTF-8 before comparing. Or maybe easier-> https://docs.microsoft.com/en-us/dotnet ... work-4.7.2

magictrip
Posts: 247
Joined: Tue, 25. Dec 18, 01:04

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by magictrip » Tue, 8. Jan 19, 22:34

nonnex wrote:
Tue, 8. Jan 19, 22:02
convert haystack and needle strings to to UTF-8 before comparing. Or maybe easier-> https://docs.microsoft.com/en-us/dotnet ... work-4.7.2
Thanks for the tip, I will look into that.

Baldamundo
Posts: 99
Joined: Sat, 21. Jun 08, 18:46
x4

Re: [TOOL] X4LogAnalyzer - Help analyse the trade log

Post by Baldamundo » Thu, 10. Jan 19, 07:09

This looks fantastic, but I'm getting a lot of error spam saying "Error while trying to summarize the operations - Object reference not set to an instance of an object.", and a lot of ships aren't showing up in the list, and the graphs aren't displaying properly.

Could it be some sort of mod conflict? And if so, any ideas what type of mod's likely to be the culprit?

Post Reply

Return to “X4: Foundations - Scripts and Modding”