[TOOL] X4LogAnalyzer - Help analyse the trade log
Moderators: Moderators for English X Forum, Scripting / Modding Moderators
[TOOL] X4LogAnalyzer - Help analyse the trade log
DISCLAIMER
In order to be able to understand a little bit better what my ships were doing I developed a little application called X4LogAnalyzer.
The goal for this application is to extract the information from the X4 Save game log and put it in a way that is easier to understand. It in no way harm your save game as it will make a copy and then work with the copy and it also do not alter anything inside of it (so you can still venture), but nevertheless I suggest to do a backup before using the application.
SOURCE CODE
The source code can be downloaded here:
https://github.com/magictripgames/X4Mag ... ogAnalyzer
HOW TO INSTALL/USE THE APPLICATION
The application requires .Net Framework 4.6.1 and does not requires installation. To use it simple download it from :
https://drive.google.com/open?id=1S5YQ2 ... zy2979c-ZT
Unzip it and look for the file called X4LogAnalyzer.exe and run it. You will see the Operations Window:
In the center of the window you have 2 buttons:
"Select Log File" - this button is used to select and import the information's from your save file. Simple click it and enter the full path of the save file you want to import and hit OK.
After you hit OK your save will start to be imported. This operation takes a couple of seconds and the application will be irresponsible during it (sorry, still working on it). After the import is complete you can click on other tabs to analyze the information.
"Clear Cached Data" - When you import the log file a file called X4LogAnalyzerTempXML.json is created and it stores all the information extracted from the save file for later usage. The Clear Cached Data when clicked will delete this file so you can start a new process. Usually this is used when you started a new save and need to clean the old data.
After you imported your Save File you can now use the application to analyze the information using it's 3 main functionalities:
SHIP INFORMATION:
The goal of this functionality is to show which transactions were imported for each ship/station. It's columns are:
Time: When the even occurred
Ship ID: The ID of the ship that generated the event.
Ship Name: It's the full name of the ship. If you changed the ship name during your gameplay the last name should be the one displayed here.
Product: What was the Ware that was sold by that ship
Destination: To whom the Ware was sold
Quantity: How many items were sold
Item Price: For how much each item was sold
Total: Quantity * Item Price
Estimated Profit: (Item Price - Ware Min Price) * Quantity
Ship Inventory used: Quantity * Ware Weight
You will also be able to see the Time in Service, which mean the amount of time from the first trade operation to the last, Items Traded is the total of items traded by that ship/station and Total Value Traded is the total of money received by that ship (not the profit).
SHIP ANALYSIS:
This functionality is used to compare the ships/stations to check which one is bringing more money to our empire. For each ship you can add 4 different "pens":
Show Individual Sales - Off and Deduce Estimate Ware Cost - OFF: This will bring the accumulated money received over time, basically an accumulated of what is been reported on the log. This is good when you are checking miners, for example as their material cost is 0 or for Stations that produce the material.
Show Individual Sales - Off and Deduce Estimate Ware Cost - On: This is the estimated accumulated profit. This is good for traders as they only report the final sale value, but the actual profit.
Show Individual Sales - On and Deduce Estimate Ware Cost - Off: This will show how much money was received in each sale and it's useful to indicate Wares that are losing value over time (For example, to indicate that a market is saturated, like Silicon)
Show Individual Sales - On and Deduce Estimate Ware Cost - On: This will show how much profit you are making in each transaction. Again a decline here may indicate a market stagnation or a lack of supply.
To add a pen to the graph simple select which option you want above and then double click the ship. You can add multiple ships or the same ship with different visions. To clean the graph just select a different tab on top and return to this tab.
WARE ANALYSIS:
This functionality is used to show the performance of each Ware. It will only show the Wares that are been traded in your empire and it have 3 views:
Show Estimated Profit: Will order the wares according to the estimated profit made by each one. Again this is useful for the Trades where you are buying from the other factions.
Show Full Money Earned: Will show how much money was reported by a given Ware.
Shot Total Items Sold: Will show how many items of each ware were sold by the empire.
If you click in one of the columns it will show which ships reported selling that specific ware. This is particularly useful to check the performance of each ship in each specific ware.
To go back to the graph just click on the respective selection box.
I appreciate any feedback and suggestions.
In order to be able to understand a little bit better what my ships were doing I developed a little application called X4LogAnalyzer.
The goal for this application is to extract the information from the X4 Save game log and put it in a way that is easier to understand. It in no way harm your save game as it will make a copy and then work with the copy and it also do not alter anything inside of it (so you can still venture), but nevertheless I suggest to do a backup before using the application.
SOURCE CODE
The source code can be downloaded here:
https://github.com/magictripgames/X4Mag ... ogAnalyzer
HOW TO INSTALL/USE THE APPLICATION
The application requires .Net Framework 4.6.1 and does not requires installation. To use it simple download it from :
https://drive.google.com/open?id=1S5YQ2 ... zy2979c-ZT
Unzip it and look for the file called X4LogAnalyzer.exe and run it. You will see the Operations Window:
In the center of the window you have 2 buttons:
"Select Log File" - this button is used to select and import the information's from your save file. Simple click it and enter the full path of the save file you want to import and hit OK.
After you hit OK your save will start to be imported. This operation takes a couple of seconds and the application will be irresponsible during it (sorry, still working on it). After the import is complete you can click on other tabs to analyze the information.
"Clear Cached Data" - When you import the log file a file called X4LogAnalyzerTempXML.json is created and it stores all the information extracted from the save file for later usage. The Clear Cached Data when clicked will delete this file so you can start a new process. Usually this is used when you started a new save and need to clean the old data.
After you imported your Save File you can now use the application to analyze the information using it's 3 main functionalities:
SHIP INFORMATION:
The goal of this functionality is to show which transactions were imported for each ship/station. It's columns are:
Time: When the even occurred
Ship ID: The ID of the ship that generated the event.
Ship Name: It's the full name of the ship. If you changed the ship name during your gameplay the last name should be the one displayed here.
Product: What was the Ware that was sold by that ship
Destination: To whom the Ware was sold
Quantity: How many items were sold
Item Price: For how much each item was sold
Total: Quantity * Item Price
Estimated Profit: (Item Price - Ware Min Price) * Quantity
Ship Inventory used: Quantity * Ware Weight
You will also be able to see the Time in Service, which mean the amount of time from the first trade operation to the last, Items Traded is the total of items traded by that ship/station and Total Value Traded is the total of money received by that ship (not the profit).
SHIP ANALYSIS:
This functionality is used to compare the ships/stations to check which one is bringing more money to our empire. For each ship you can add 4 different "pens":
Show Individual Sales - Off and Deduce Estimate Ware Cost - OFF: This will bring the accumulated money received over time, basically an accumulated of what is been reported on the log. This is good when you are checking miners, for example as their material cost is 0 or for Stations that produce the material.
Show Individual Sales - Off and Deduce Estimate Ware Cost - On: This is the estimated accumulated profit. This is good for traders as they only report the final sale value, but the actual profit.
Show Individual Sales - On and Deduce Estimate Ware Cost - Off: This will show how much money was received in each sale and it's useful to indicate Wares that are losing value over time (For example, to indicate that a market is saturated, like Silicon)
Show Individual Sales - On and Deduce Estimate Ware Cost - On: This will show how much profit you are making in each transaction. Again a decline here may indicate a market stagnation or a lack of supply.
To add a pen to the graph simple select which option you want above and then double click the ship. You can add multiple ships or the same ship with different visions. To clean the graph just select a different tab on top and return to this tab.
WARE ANALYSIS:
This functionality is used to show the performance of each Ware. It will only show the Wares that are been traded in your empire and it have 3 views:
Show Estimated Profit: Will order the wares according to the estimated profit made by each one. Again this is useful for the Trades where you are buying from the other factions.
Show Full Money Earned: Will show how much money was reported by a given Ware.
Shot Total Items Sold: Will show how many items of each ware were sold by the empire.
If you click in one of the columns it will show which ships reported selling that specific ware. This is particularly useful to check the performance of each ship in each specific ware.
To go back to the graph just click on the respective selection box.
I appreciate any feedback and suggestions.
Last edited by magictrip on Wed, 9. Jan 19, 00:46, edited 3 times in total.
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
FAQs:
- My game is in <INSERT LANGUAGE HERE>, can I still use this application?
A> I only tested the application with the English log and as it relies on a text based log, it does not work by default. I included 2 config files, one called Wares.json and the other Configuration.Json (this one is generated the first time you run the application) which contains the names of the wares in English and the log markers I'm using respectively. So, if you want to try you can translate the Ware names and also translate the expressions: sold, to, in and Trade Complete that are in those files. After that if the application still do not work please provide the translated files alongside a Save file in your language and I will try to "internationalize" it.
- The application runs fine but when I try to import the log nothing happens
A> The application relies on the logs generated by the Vanilla game, some mods disable those logs so if that is happening the application will not be able to load the log files. Also, if you hit the "Clean log" button inside the game the save game log will be also deleted and the application also cannot recover that. If you have already imported the log then you can hit clean log as the application will keep a history of what it already imported.
Known bugs:
Sometimes the application will fail to clean the temporary XML file created to import the saves. To solve this problem just close the application and delete the file manually.
Change Log:
- v 0.1 - Initial Realease
- v 0.2 - Fixed bug on Ware Analysis where if you load a new save file it was not keeping the cached information
- v 0.3 - The application will not show a message informing the error in case it cannot process the selected Save File
- v 0.4 - Added PARTIAL EXPERIMENTAL support for TaterTrade Mod. This support was not discussed with the Mod maker so it may break at any time and/or not be fully compatible.
- v 0.5 - Fixed issue with not taking into account translated string length.
Fixed issue where SoldTranslation config were been saved everytime
Added German reference files - Thanks to @P.Hastig
- v 0.6 - Fixed an issue with an unknown mod adding a (xxx Cr) after the ware name which was making it not possible to find the wares
Changed the wares tab behavior to now show also wares that have a negative income, this would only be shown if some mods are active (don't know which one though)
- My game is in <INSERT LANGUAGE HERE>, can I still use this application?
A> I only tested the application with the English log and as it relies on a text based log, it does not work by default. I included 2 config files, one called Wares.json and the other Configuration.Json (this one is generated the first time you run the application) which contains the names of the wares in English and the log markers I'm using respectively. So, if you want to try you can translate the Ware names and also translate the expressions: sold, to, in and Trade Complete that are in those files. After that if the application still do not work please provide the translated files alongside a Save file in your language and I will try to "internationalize" it.
- The application runs fine but when I try to import the log nothing happens
A> The application relies on the logs generated by the Vanilla game, some mods disable those logs so if that is happening the application will not be able to load the log files. Also, if you hit the "Clean log" button inside the game the save game log will be also deleted and the application also cannot recover that. If you have already imported the log then you can hit clean log as the application will keep a history of what it already imported.
Known bugs:
Sometimes the application will fail to clean the temporary XML file created to import the saves. To solve this problem just close the application and delete the file manually.
Change Log:
- v 0.1 - Initial Realease
- v 0.2 - Fixed bug on Ware Analysis where if you load a new save file it was not keeping the cached information
- v 0.3 - The application will not show a message informing the error in case it cannot process the selected Save File
- v 0.4 - Added PARTIAL EXPERIMENTAL support for TaterTrade Mod. This support was not discussed with the Mod maker so it may break at any time and/or not be fully compatible.
- v 0.5 - Fixed issue with not taking into account translated string length.
Fixed issue where SoldTranslation config were been saved everytime
Added German reference files - Thanks to @P.Hastig
- v 0.6 - Fixed an issue with an unknown mod adding a (xxx Cr) after the ware name which was making it not possible to find the wares
Changed the wares tab behavior to now show also wares that have a negative income, this would only be shown if some mods are active (don't know which one though)
Last edited by magictrip on Fri, 11. Jan 19, 18:46, edited 6 times in total.
-
- Posts: 11
- Joined: Tue, 4. Dec 18, 16:30
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Sounds like a great tool @magictrip. I have a german savegame file. If I run your tool, I got no output due to the missing translations I guess. So I used as suggested the Configurations.json like
[{"Key":"LastSaveGameLoaded","Value":"C:\\Users\\XY\\Documents\\Egosoft\\X4\\XXXXXXXXX\\save\\save_002.xml.gz"},{"Key":"SoldTranslation","Value":"verkaufte"},{"Key":"TradeCompletedTranslation","Value":"Auftrag abgeschlossen"},{"Key":"InTranslation","Value":"in"},{"Key":"ToTranslation","Value":"an"},{"Key":"SoldTranslation","Value":"sold"}]
but the LogAnalyzer crashes then without error message. Did I miss something?
[{"Key":"LastSaveGameLoaded","Value":"C:\\Users\\XY\\Documents\\Egosoft\\X4\\XXXXXXXXX\\save\\save_002.xml.gz"},{"Key":"SoldTranslation","Value":"verkaufte"},{"Key":"TradeCompletedTranslation","Value":"Auftrag abgeschlossen"},{"Key":"InTranslation","Value":"in"},{"Key":"ToTranslation","Value":"an"},{"Key":"SoldTranslation","Value":"sold"}]
but the LogAnalyzer crashes then without error message. Did I miss something?
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Well, I don't really know, maybe the Wares need to be translated as well. Could you please provide a Save game so I can check what is exactly the error that is happening?Perry-Rhodan49 wrote: ↑Sat, 29. Dec 18, 07:58Sounds like a great tool @magictrip. I have a german savegame file. If I run your tool, I got no output due to the missing translations I guess. So I used as suggested the Configurations.json like
[{"Key":"LastSaveGameLoaded","Value":"C:\\Users\\XY\\Documents\\Egosoft\\X4\\XXXXXXXXX\\save\\save_002.xml.gz"},{"Key":"SoldTranslation","Value":"verkaufte"},{"Key":"TradeCompletedTranslation","Value":"Auftrag abgeschlossen"},{"Key":"InTranslation","Value":"in"},{"Key":"ToTranslation","Value":"an"},{"Key":"SoldTranslation","Value":"sold"}]
but the LogAnalyzer crashes then without error message. Did I miss something?
PS: I can provide the source code as well if you want to debug it yourself
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
The application just crashes for me after inputting my save file. Do i need to install the "Application Manifest"? I already have the .net framework installed.
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Not really, I tested on my son computer and it worked fine. Can you provide your save file so I can check what is the problem?
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Ok, just uploaded a new version and it now should show the error message while importing the file, you can download it here:
https://drive.google.com/open?id=1S5YQ2 ... zy2979c-ZT
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Great tool - helped me to quickly identify which products are making the best margins. Am going to setup a profit focused trade fleet to target these products.. leaving my existing traders to trade every legal ware so that they keep the economy going a bit.
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Good to know it helped you, my goal was exactly the same as you
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Today, I just entered forum to look for something like this. Great work my friend.
I am testing data of my existing station and want to ask about data. Is it possible distinguish a trade sell and buy. Some items from my self sufficient station show 0 profit and I am puzzled about this. I am guessing some of them can be a buy but not all
I am testing data of my existing station and want to ask about data. Is it possible distinguish a trade sell and buy. Some items from my self sufficient station show 0 profit and I am puzzled about this. I am guessing some of them can be a buy but not all
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
I can only see what the game logs, sadly every time you sell to yourself, to your own ships or a station subordinate sells the ware they will not generate a log.yafeshan wrote: ↑Tue, 1. Jan 19, 09:14Today, I just entered forum to look for something like this. Great work my friend.
I am testing data of my existing station and want to ask about data. Is it possible distinguish a trade sell and buy. Some items from my self sufficient station show 0 profit and I am puzzled about this. I am guessing some of them can be a buy but not all
So, if your station is selling to other factions directly, that will show in the log and you will see it in the application. Also, if you set your Auto Traders manually instead of been station subordinates it will also show that ship selling the item. It's not perfect but works just fine.
Also, the game does not log how much you paid for a ware, that is why the profit is an estimation based on the min price, although a lot of times you are paying more than the min price.
-
- Posts: 40
- Joined: Tue, 25. Sep 18, 23:57
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
It doesn't work with non gziped save files.
-
- Posts: 28
- Joined: Sat, 31. Jan 09, 20:20
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Hi
Thanks for building this! With the X games so focused on economy simulation you would think by now we would have at least some clear and useful reporting in-game!
However the analyser seams to have stopped working for me at the import stage, all was working fine then I Installed the below mod to throw in some extra log messages in but it seams to have broken the import process of the Analyser. Not sure if you could add compatibility for the below message structures and/or bypass any ones that the analyser doesn't pick up correctly (assuming this mod is the cause of the error and its not just a coincidence).
https://www.nexusmods.com/x4foundations ... escription
The logs the mod adds appear to be in the format
[Ship Name] [Ship ID] bought [amount] [ware] from [Station Name] [Station ID] in [Sector] for [Credits]
Thanks again for a great tool!
Thanks for building this! With the X games so focused on economy simulation you would think by now we would have at least some clear and useful reporting in-game!
However the analyser seams to have stopped working for me at the import stage, all was working fine then I Installed the below mod to throw in some extra log messages in but it seams to have broken the import process of the Analyser. Not sure if you could add compatibility for the below message structures and/or bypass any ones that the analyser doesn't pick up correctly (assuming this mod is the cause of the error and its not just a coincidence).
https://www.nexusmods.com/x4foundations ... escription
The logs the mod adds appear to be in the format
[Ship Name] [Ship ID] bought [amount] [ware] from [Station Name] [Station ID] in [Sector] for [Credits]
Thanks again for a great tool!
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
magictrip wrote: ↑Sat, 29. Dec 18, 21:35Perry-Rhodan49 wrote: ↑Sat, 29. Dec 18, 07:58Sounds like a great tool @magictrip. I have a german savegame file. If I run your tool, I got no output due to the missing translations I guess. So I used as suggested the Configurations.json like
[{"Key":"LastSaveGameLoaded","Value":"C:\\Users\\XY\\Documents\\Egosoft\\X4\\XXXXXXXXX\\save\\save_002.xml.gz"},{"Key":"SoldTranslation","Value":"verkaufte"},{"Key":"TradeCompletedTranslation","Value":"Auftrag abgeschlossen"},{"Key":"InTranslation","Value":"in"},{"Key":"ToTranslation","Value":"an"},{"Key":"SoldTranslation","Value":"sold"}]
but the LogAnalyzer crashes then without error message. Did I miss something?
Well, I don't really know, maybe the Wares need to be translated as well. Could you please provide a Save game so I can check what is exactly the error that is happening?
PS: I can provide the source code as well if you want to debug it yourself
Hi magictrip,
i also have a german savegame, so i translated both files (Configurations.json and Wares.json). Unfortunately your tool still crashes with exception:
Error found while trying to import the save game. Error description: Die Eingabezeichenfolge hat das falsche Format. (translated: the input string has the wrong format).
I tried to compile and debug your source but too many assemblies are missing (e.g. AvalonEdit, LiveCharts..).
Here're the config files and the savegame:
https://drive.google.com/drive/folders/ ... sp=sharing
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Hello P.Hastig. I was able to find a bug that was introduced when I added support for TaterTrade. Please download the new version and it should work for you now.P.Hastig wrote: ↑Sun, 6. Jan 19, 23:31Hi magictrip,
i also have a german savegame, so i translated both files (Configurations.json and Wares.json). Unfortunately your tool still crashes with exception:
Error found while trying to import the save game. Error description: Die Eingabezeichenfolge hat das falsche Format. (translated: the input string has the wrong format).
I tried to compile and debug your source but too many assemblies are missing (e.g. AvalonEdit, LiveCharts..).
Here're the config files and the savegame:
https://drive.google.com/drive/folders/ ... sp=sharing
Please let me know if you are still having problems.
PS: I added your files in the new version so other people can use it if needed
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Thank you,magictrip wrote: ↑Mon, 7. Jan 19, 01:34Hello P.Hastig. I was able to find a bug that was introduced when I added support for TaterTrade. Please download the new version and it should work for you now.
Please let me know if you are still having problems.
PS: I added your files in the new version so other people can use it if needed
the new version now can handle german savegames, but there is still a little issue.
It seems, that product-strings with special characters like ä, ö, ü can't be parsed.
That's strange, because destination-strings like "ARG Hüllenteilefabrik I" where parsed correctly.
See here..:
..and the corresponding rows in the savegame file:
Code: Select all
<entry time="147056.314" title="Handel abgeschlossen" text="Hermes Angreifer QRT-394 verkaufte 50 Sensorenbündel an ARG Argon Raumdock JCB-827 in Argon Prime für 104074 Cr." faction="{20203,201}" money="10407450"/>
<entry time="150377.804" title="Handel abgeschlossen" text="Hermes Angreifer QRT-394 verkaufte 10 Sensorenbündel an ARG Geschützturmkomponenten-Fabrik I DBX-715 in Argon Prime für 18861 Cr." faction="{20203,201}" money="1886130"/>
Also i made a little mistake translating wares.json in the first row. The char 'e' at the end was missing: "Name": "Fortschrittliche Verbundwerkstoffe".
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Well, I believe the difference there is that for ship names I use the identifier (like QRT-394) to do the queries and searches, while the wares I use the name, I may have to remove special characters or something like that for them to behave correctly. So, I will need to check that.P.Hastig wrote: ↑Tue, 8. Jan 19, 20:44Thank you,
the new version now can handle german savegames, but there is still a little issue.
It seems, that product-strings with special characters like ä, ö, ü can't be parsed.
That's strange, because destination-strings like "ARG Hüllenteilefabrik I" where parsed correctly.
See here..:
..and the corresponding rows in the savegame file:It's a newer savegame (save_005.xml.gz), i upoaded it to the same directory.Code: Select all
<entry time="147056.314" title="Handel abgeschlossen" text="Hermes Angreifer QRT-394 verkaufte 50 Sensorenbündel an ARG Argon Raumdock JCB-827 in Argon Prime für 104074 Cr." faction="{20203,201}" money="10407450"/> <entry time="150377.804" title="Handel abgeschlossen" text="Hermes Angreifer QRT-394 verkaufte 10 Sensorenbündel an ARG Geschützturmkomponenten-Fabrik I DBX-715 in Argon Prime für 18861 Cr." faction="{20203,201}" money="1886130"/>
Also i made a little mistake translating wares.json in the first row. The char 'e' at the end was missing: "Name": "Fortschrittliche Verbundwerkstoffe".
PS: Thanks for the fixed wares.json, I will upload to the new version as well.
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
convert haystack and needle strings to to UTF-8 before comparing. Or maybe easier-> https://docs.microsoft.com/en-us/dotnet ... work-4.7.2
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
Thanks for the tip, I will look into that.nonnex wrote: ↑Tue, 8. Jan 19, 22:02convert haystack and needle strings to to UTF-8 before comparing. Or maybe easier-> https://docs.microsoft.com/en-us/dotnet ... work-4.7.2
-
- Posts: 99
- Joined: Sat, 21. Jun 08, 18:46
Re: [TOOL] X4LogAnalyzer - Help analyse the trade log
This looks fantastic, but I'm getting a lot of error spam saying "Error while trying to summarize the operations - Object reference not set to an instance of an object.", and a lot of ships aren't showing up in the list, and the graphs aren't displaying properly.
Could it be some sort of mod conflict? And if so, any ideas what type of mod's likely to be the culprit?
Could it be some sort of mod conflict? And if so, any ideas what type of mod's likely to be the culprit?