Analysis attempt of all trades in my savefile

[Browser_ice]

Hi, I have been trying to analyse all the buying and selling trade offers stored in my savefile. It is an attempt to see where are the profits and trends.

I managed to extract them and loaded them into Notepad++ to remove the unnecessary stuff and loaded it into LibreOffice Calc for analysis.

I wanted to do graphs for all or each wares but due to the amount of data, it is taking for ever. Therefore I had to resort to a simpler way until I can get what I want.

The total amount of data rows for buying trades is 228,658 and for the selling trades, 127,773. The ODS file is 19Meg.

[added comment] Since every new game is randomly generated, this is strickly related to my current game session and may not represent anybody else's game session.[/added comment]

Here are screenshots of each trade kinds summaries:

http://pasteall.org/pic/show.php?id=e6a ... 8d5cbcd587

http://pasteall.org/pic/show.php?id=933 ... adcf19bf55

I had a few surprises when I looked at them.

Any ideas on what I could use to do a trend analysis that won't take an hour ?

I was thinking about writing my own software to do it (Java or something else). That would be my first programming attempt. Haven't done any programming in 20 years and I used to know a dozen languages. I would just have to commit doing it ....


{Images posted directly to the forums should not be greater than 640x480 or 100kb, oversize image now linked - Terre}

[Dirk-Jan]

I was thinking about that but got stuck at the first step: getting the game log into a txt file. Would you be willing to share how you did that?

[Browser_ice]

I used Notepad++ to open the savefile and isolate the <economylog>. ... </economylog> section into a new text file

I then isolated only the buying and selling trades (not the summary)

Then using OpenOffice Calc, I divided that text into columns. I had to move a few columns as some records did not have all the variables as others.

I then sorted out based on trade type (buying or selling), wares and time. I used a few Calc Database commands to get the totals.

Initially I wanted to draw graphs but it was just toooooo loooonnngggg due to too many records.

[Imperial Good]

One can also parse save files into DOM form using some simple hacked together Java code. This can allow one to programmatically filter records and potentially output data in a more user friendly or condensed way.

Simple analysis might not be too useful unless you also take into consideration the factions involved. This is because HOP economy is isolated and one generally does not want to touch HAT or SCA trade offers due to the pirating risk. It may also incorrectly detect Xenon construction orders for Energy Cells, Ore and Silicon.

[Dirk-Jan]

I used Notepad++ to open the savefile and isolate the <economylog>. ... </economylog> section into a new text file

I then isolated only the buying and selling trades (not the summary)

Then using OpenOffice Calc, I divided that text into columns. I had to move a few columns as some records did not have all the variables as others.

I then sorted out based on trade type (buying or selling), wares and time. I used a few Calc Database commands to get the totals.

Initially I wanted to draw graphs but it was just toooooo loooonnngggg due to too many records.

Thanks for sharing