Hint about next patch from Egosoft? CBJ just posted this...

[DaddyMonster]

This would seem to suggest that the next patch may include a 64 bit .exe (which could potentially bring MAJOR performance improvements).

I think thats highly doubful, as something like a 64bit .exe would take them ages at the current progress. Making XR into a 64bit app wouldnt also make the game faster; there is no reason why a 32bit app couldnt run fast.

The problem is that .exe works on .XML files in runtime, which is a huge drain in resources (as found by people). X-Rebirth is just poorly coded. Even on my site I notice around x300 times increase in load-times when loading from cached pages, compared to building the page with XR in-game XML's game-files. If your CPU in-game is working on 1000's XML's per second, then it will be slow. XR needs to pre-load the XML, then convert them into an array. If the x300 rule works the same, then this would automatically make XR run around 300fps+. When looking at the game, I cant see why XR would actually be much stress for processor. When you look at XR, how can it use multiple cores (unlike X3AP/TC, for me 4 vs 1) but yet run 10th of the speed (and x1000 slower when SETA is at x10 in X3AP).... techially for me X3AP runs fine with 3.4Ghz worth CPU, yet XR with 13.6Ghz CPU (max) is slower. XR is wasting a lot of my CPU grunt.

Yes X3AP used XML, but only for MD, universe, scripting and a few odd things (not all the time). Nothing that demanding. The rest of it (like ships, wares, lasers, bullets etc.) were loaded in from a CSV file (/types/) which is easy to convert into a array. If this is the case, then doesnt fill me much confidence. Performance lost like this are easily to spot, and shouldnt been in release version of the game.

Sure, but there's a limit to how many assets you can preload - which is where the move to 64bit comes in as memory is limited to under four gig. Even if you're not maxing out memory, a sudden requirement to load assets will tax the processor. Handling asynchronous preloading can be a good solution depending on the context and implementation. Sometimes there are just too many xml docs to pre-parse them all and it's impossible to predict which you will need. I suspect the current memory just isn't cutting it so they likely want to go to more memory and preload more. The improvement could be dramatic but that's based on lots of guesswork.

ps. does it really take less resources to parse csv into an array than xml?? The point might be xml tends to contain everything and the csv is divided into discrete data sets so maybe that's it.

[BigBANGtheory]

more fleets command are in the next beta patch or not?

I'd settle for an open discussion on the topic at this point.

[Sam L.R. Griffiths]

ps. does it really take less resources to parse csv into an array than xml?? The point might be xml tends to contain everything and the csv is divided into discrete data sets so maybe that's it.

In short, yes - but XML is more flexible and powerful as a data format - and with the right editors more user friendly. XML need not be costly to process though, the key is to picking your XML parser appropriately and not un-necessarily processing the XML in text form but rather keeping it in the parser's DOM form where possible. In addition, modern CPUs have special commands to assist with the parsing of XML data (which includes HTML I should add).

[khartsh]

ps. does it really take less resources to parse csv into an array than xml?? The point might be xml tends to contain everything and the csv is divided into discrete data sets so maybe that's it.

In short, yes - but XML is more flexible and powerful as a data format - and with the right editors more user friendly. XML need not be costly to process though, the key is to picking your XML parser appropriately and not un-necessarily processing the XML in text form but rather keeping it in the parser's DOM form where possible. In addition, modern CPUs have special commands to assist with the parsing of XML data (which includes HTML I should add).

The csv files before were used almost as read only with no modification to assets... it read what the stats for a Buster was, it didn't save information about a Buster to them.

The XML files today are being used for dynamic data and random generation of assets... CBJ can say he is wrong all he wants but if you look at the actual work the CPU Threads are doing, most of it involves parsing, cleaning and reading XML files or waiting for assets to become available for the next thread to do the same.

[CBJ]

I see we are back to certain people thinking they know better and dismissing everything anyone from Egosoft says as lies or incompetence. Under the circumstances there is very little point in me engaging further in this thread.

[swatti]

Angry non-professional fans ALLWAYS know best solution to everything! ^^

Still. ONE QUESTION STANDS and we deserve an answer.

"Fleet commands, when?"

This alone could be considored a "content patch" as it gives us something to do with things we have and can build. While there are station manager AI-issues etc, they work "somewhat" but OUR (COMBAT)SHIPS DO NOT WORK, AT ALL.

[khartsh]

I see we are back to certain people thinking they know better and dismissing everything anyone from Egosoft says as lies or incompetence. Under the circumstances there is very little point in me engaging further in this thread.

I am not addressing you as a moderator nor as an employee of Egosoft. I want the CBJ the gamer answer.

Profilers have posted evidence contrary to your previous post, to the effect that 2/3 of all CPU utilization revolve around those XML files in some way on the main thread. Do you have a profile report that contradicts that?

[graphicboy]

I see we are back to certain people thinking they know better and dismissing everything anyone from Egosoft says as lies or incompetence. Under the circumstances there is very little point in me engaging further in this thread.

I am not addressing you as a moderator nor as an employee of Egosoft. I want the CBJ the gamer answer.

Profilers have posted evidence contrary to your previous post, to the effect that 2/3 of all CPU utilization revolve around those XML files in some way on the main thread. Do you have a profile report that contradicts that?

I have not looked into those posts, nor have I profiled the application myself.

I can tell you however that both internal AND external profilers can be extremely misleading because they get in the way of execution of the threads they're profiling.

[Edit:] In fact I've seen this happen on XML loading processing more than anything else. Turned out that sans profiler the XML loading finished in less than a second and the actual problem was somewhere completely unrelated. [/Edit]

They are meaningless without diving into the code itself and toying with it until the truth unveils itself. Profilers are simply a clue, not the answer.

To say otherwise is to show your ignorance, as is saying CBJ doesn't know what's going on. I guarantee you as a senior developer myself that whatever profilers the gamers are running are a waste of time.

[Roguey]

I see we are back to certain people thinking they know better and dismissing everything anyone from Egosoft says as lies or incompetence. Under the circumstances there is very little point in me engaging further in this thread.

Well I thank-you for explaining, I just put two and two together and thought that it was the XML causing the slowdown's. If youre saying thats not the cause, then I believe you CBJ. Thanks.

[Santi]

I see we are back to certain people thinking they know better and dismissing everything anyone from Egosoft says as lies or incompetence. Under the circumstances there is very little point in me engaging further in this thread.

Well I thank-you for explaining, I just put two and two together and thought that it was the XML causing the slowdown's. If youre saying thats not the cause, then I believe you CBJ. Thanks.

That quote has nothing to do with you Roguey, CBJ post was referring to Kharths post. CBJ was referring to your posts in:

"The work alluded to in that post is related to reducing some memory overheads, not to the 64-bit version. We are, of course, working on the 64-bit version, as Bernd stated in the most recent update, but that's not slated for release as part of the next patch.
Regarding XML files, Roguey, I'm afraid you are quite simply wrong. Of course XML files are loaded and converted into internal formats for use in-game. We are not continuously reading data direct from XML thousands of times per frame"

and:

"The poster did say that that data needed to be taken with a pinch of salt, and he's right. I'm not going to go into detail but asynchronously loading files on asset management worker threads is pretty standard stuff.
Where you went wrong after that is when you made the leap from the game loading XML files to it continuously referencing data contained in those files in XML form every time it wants something from them, which quite simply isn't the case"

[ghostpilot]

So in short, this all means the new x64 executable will not be the ultimate fix for the bad fps? great.

Any word on getting SLI to actually work? my card works almost to the point of melting without the help of the scond one.

[spankahontis]

Nearly the end of the month so I take it your squeezing out a beta for Friday/Saturday?
Or Next week?

[khartsh]

I see we are back to certain people thinking they know better and dismissing everything anyone from Egosoft says as lies or incompetence. Under the circumstances there is very little point in me engaging further in this thread.

Well I thank-you for explaining, I just put two and two together and thought that it was the XML causing the slowdown's. If youre saying thats not the cause, then I believe you CBJ. Thanks.

That quote has nothing to do with you Roguey, CBJ post was referring to Kharths post. CBJ was referring to your posts in:

"The work alluded to in that post is related to reducing some memory overheads, not to the 64-bit version. We are, of course, working on the 64-bit version, as Bernd stated in the most recent update, but that's not slated for release as part of the next patch.
Regarding XML files, Roguey, I'm afraid you are quite simply wrong. Of course XML files are loaded and converted into internal formats for use in-game. We are not continuously reading data direct from XML thousands of times per frame"

and:

"The poster did say that that data needed to be taken with a pinch of salt, and he's right. I'm not going to go into detail but asynchronously loading files on asset management worker threads is pretty standard stuff.
Where you went wrong after that is when you made the leap from the game loading XML files to it continuously referencing data contained in those files in XML form every time it wants something from them, which quite simply isn't the case"

I think Roguey was responding to the post where CBJ said he was "Wrong"

[graphicboy]

Seems based on the time of day that unless everyone's working late - yet another weekend gone...

Sad thing is despite all my searching I haven't found anything else that does what XR is supposed to do. Guess I'll keep banging refresh until something comes through.

[nalim27]

I really do not understand why Egosoft can't (or wouldn't) inform us - customers - more often.
For example why in some topic (maybe in beta forum) at end of week some developer did not write progress about what is done?
Something like this:

WEEK 1:
• Added logbook entries for refuelling, trade and building.
• Added call from manager when a station is low on credits.
• Added ship name when player gets a call requiring a response from a player owned ship.
• Fixed a cause of occasional crashes when loading savegame.
• Fixed a cause of occasional crashes during large battles.

WEEK2:
• New Feature: Smalltalk reward allowing you to receive trade offer updates for a station remotely. • Added logbook entries for refuelling, trade and building.
• Added call from manager when a station is low on credits.
• Added ship name when player gets a call requiring a response from a player owned ship.
• Fixed a cause of occasional crashes when loading savegame.
• Fixed a cause of occasional crashes during large battles.
• Fixed another cause of the game hanging after Alt-Tab.
• Added prices for small ships so that these can now be sold.
• Cargo value is now taken into account when selling a ship; you get roughly half it's value.
• Fixed price calculations for intermediate wares.
• Fixed two causes of disappearing ships and one of ships ending up in strange places.

WEEK3:
• New Feature: Player-owned ships and stations can now be renamed.
• New Feature: Smalltalk reward allowing you to receive trade offer updates for a station remotely.
• Added logbook entries for refuelling, trade and building.
• Added call from manager when a station is low on credits.
• Added ship name when player gets a call requiring a response from a player owned ship.
• Fixed a cause of occasional crashes when loading savegame.
• Fixed a cause of occasional crashes during large battles.
• Fixed several other causes of occasional crashes.
• Fixed another cause of the game hanging after Alt-Tab.
• Added prices for small ships so that these can now be sold.
• Cargo value is now taken into account when selling a ship; you get roughly half it's value.
• Fixed price calculations for intermediate wares.
• Fixed two causes of disappearing ships and one of ships ending up in strange places.
• Fixed certain station elements being rotated incorrectly after loading a savegame.
• Fixed problem with random station elements being built and preventing further building.
• Fixed problem with small ships preventing building.
• Fixed problem with drones not being able to dock at certain mining ships.
• Fixed incorrect container amount being dropped from cargo drones.
• Fixed another issue with cargo collection.
• Fixed buy offers for more wares than can be stored.
• Fixed player being able to start multiple boarding operations on the same ship.
• Fixed abort button for boarding operations.
• Fixed info and comm options for externally docked ships.
• Fixed issue with operational range settings resulting in ships looking for trades in the wrong places.
• Improved station-owned mining ship AI to prevent station from filling with more resources than it wants.
• Improved shooting logic for NPC ships (more improvements to come).
• Fixed a problem resulting in inactive patrol ships.
• Fixed a problem with NPC beam weapons not firing correctly.

And so on.

Simply said - inform us about weekly progress on bug fixing, feature added etc. It will not take too much time - every developer company must create changelog. So why not to publish it weekly?

Month silence from developers is very ... scary. We thinks some worst scenarios why they are silent - Egosoft is going to bankrupt? Did some core developer died in car accident? Did Russia attacked Germany (after they annexed Ukraine) and started Third World War?
If answer is no, then why they are silent?

[Ericius11]

The fact of the matter is most changelogs won't be that long. Especially right now, where most of their days are probably spend testing and re-testing features. Can you imagine how frustrated certain (nameless) forum goers may be if they looked and saw two weeks in a row of "Feature Testing?"

Honestly, I agree with you. I want to see the updates, and I'm okay with the idea that Egosoft may have accomplished nothing in a week. Because at the very least, they've spent a week finding ways to NOT improve the game. But I know a lot of people wouldn't react well to hearing that.

[Observe]

One thing to realize, is a week, or even a month isn't all that long when it comes to working on software. Hell, a whole week can go by sometimes with barely anything being accomplished in terms of final result. On the other hand, there are times when a lot gets done in one day! It's hard to predict. It's not a linear thing.

One thing we can be sure of (I think), is they are working very hard, and probably long hours, bringing improvements to us. We just have to be patient. There's not much else we can do (aside from complaining).

As far as the discussion re. continuously reading XML files, this as has already been pointed out, is not correct. XML data is read, processed, and sometimes (not always) stored in memory for future use.

[VincentTH]

I have not looked into those posts, nor have I profiled the application myself.

I can tell you however that both internal AND external profilers can be extremely misleading because they get in the way of execution of the threads they're profiling.

[Edit:] In fact I've seen this happen on XML loading processing more than anything else. Turned out that sans profiler the XML loading finished in less than a second and the actual problem was somewhere completely unrelated. [/Edit]

They are meaningless without diving into the code itself and toying with it until the truth unveils itself. Profilers are simply a clue, not the answer.

To say otherwise is to show your ignorance, as is saying CBJ doesn't know what's going on. I guarantee you as a senior developer myself that whatever profilers the gamers are running are a waste of time.

Agree. This is known as the Heisenberg effect. Any profiler always injects some modification that affects performance of the software you want to profile.

[graphicboy]

I think you meant Observer Effect. They're similar, but not the same thing. Either way - yes.

[gbjbaanb]

However, if the profiler shows long amount of time in the libxml library, then its nearly always because... its spending a lot of time there. If a profiler cannot get this right, its useless.

But, whilst I agree with CBJ that they read the XML files into memory data structures (of course they do, really, everyone does this), it doesn't mean that the worker threads aren't repeatedly reading the xml into the data strucctures (due to a bug - it checks, it noticed there's an xml file, it reads it... then it checks.. and sees an xml file and reads it... I have seen bugs where the check is supposed to only process when the file changes, but processes them regardless. this could be one of those occasions I guess)

Then there;'s the other concept that the libxml code is spending a lot of time there because its blocking, this would show up as spending a lot of time in the library, but would not show a large amount of CPU time being used. Whether this is by design or a bug is not for me to say.