[MOD IDEA] X3 Terran Conflict - improved performance

Observe · Post by **Observe** » Sat, 25. Oct 08, 18:03

Right now this is just an idea requiring further R&D to see whether or not it may pan out as viable. The hypothesis being that at least part of the reason there is such a significant performance degradation with TC as compared to Reunion is due to several new DirectX shaders being used for TC. While on one hand this helps TC achieve somewhat improved graphics appearance in certain respects, it may on the other hand be a cause of performance lag - requiring higher-end computer systems than were typically needed for satifactory play with Reunion.

Our team (x3dmod) is investigating the possibility of making changes to TC so that only X3-Reunion shaders are used. If our experiments prove successful, we will make available an X3-R/TC hybrid mod for those who are happy with Reunion graphics quality, but unhappy with TC performance lag.

Probably there are other reasons for the big performance difference between TC and Reunion, but if we modders can identify some of the main ones, perhaps there is something we can do about it - at least until ES has time to make changes. If this works out, we (x3dmod team) may have such a mod available in a week or so. The main reason I'm posting this now is as potential "food for thought".

Bertone · Post by **Bertone** » Sat, 25. Oct 08, 19:23

It will be a good idea, but...

Do you remember first versions of X3R? It was pretty laggy, but after some months, with the new patches, it became very smooth.

If you do that I'll be happy to try, but I think Ego will optimize TC in future patch.

Observe · Post by **Observe** » Sat, 25. Oct 08, 19:46

Bertone, I'm sure you are correct in that ES will more than likely make improvements as much as they can.

One thing I wonder about though, is the published recommended system requirements are quite a bit higher for TC than for Reunion. If in fact part of the reason for TC high system demand is due to improved graphics, then TC may never equal Reunion from a performance perspective. Certainly I suspect many people would be more than happy with Reunion-style graphics if it meant overall game performance could be better.

OOZ662 · Post by **OOZ662** » Sat, 25. Oct 08, 20:23

I definetly would.

JohanTheCoder · Post by **JohanTheCoder** » Sat, 25. Oct 08, 20:44

I would be happy too.

It lags horribly when you get hit by something which create impact-explosions. The closer those impactexplosions are the eyes, the more horrible becomes the lag.

If you got a large ship (like M7), it works better because the impactexplosions happens farther away from the eyes.

Observe · Post by **Observe** » Sat, 25. Oct 08, 20:54

JohanTheCoder wrote:I would be happy too.

Definitely Reunion "level" graphics is nothing to sneeze about - and is actually superior to TC if in order to run TC you need to reduce graphics settings.

JohanTheCoder wrote:It lags horribly when you get hit by something which create impact-explosions.

At this point I'm surmising the following possible reasons for TC performance issues:

1. Very high poly-count for certain models
2. Added effects (e.g. explosions)
3. New DirectX shaders

All this will (or won't) bear-out during our testing over the next few days. Then we'll know how practical this idea may be.

pmcorp · Post by **pmcorp** » Sat, 25. Oct 08, 21:55

One thing I think greatly contributes to a dramatically hit in performance is the fact that we have many high detailed stations put so near each other.

I expected the spaces between station to be more sparse, possibly keeping each other from visual range.

Sometimes I'm dogfighting and perforance is great when I'm facing empty space, but as soon as I have to head in the the direction of the space station, fps goes way down.

Observe · Post by **Observe** » Sat, 25. Oct 08, 22:09

pmcorp wrote:One thing I think greatly contributes to a dramatically hit in performance is the fact that we have many high detailed stations put so near each other.

Yes, I expect you are right there pmcorp. Not much we could do about that except as you suggest moving those high-detailed models further apart so that their LOD renderings could come into play. That way at least the game would only be doing full render for one or two models at any particular viewpoint.

[EDIT] Also possibly those stations are using the new shaders which may (or may not) be part of the problem. We will be looking into this as well.

Sandalpocalypse · Post by **Sandalpocalypse** » Sat, 25. Oct 08, 22:21

I think most of the performance issues in TC are caused by CPU load not graphics load. Don't know how much graphics changes will help. THere's a few things that could be done for CPU load though.

There's a lot of ships running around in Terran Conflict: increased military patrols, new sectors and more civilians to service them... but the big one here is terran space. Lots of ships, lots of sectors. There's probably a lot of room for performance trimming in the jobs file.

Observe · Post by **Observe** » Sat, 25. Oct 08, 22:28

Sandalpocalypse wrote:I think most of the performance issues in TC are caused by CPU load not graphics load. Don't know how much graphics changes will help.

You may well be right there Sandalpocalypse. We'll know soon enough. Most likely any significant "cure" will require attention on multiple fronts.

Sandalpocalypse wrote:There's a lot of ships running around in Terran Conflict: increased military patrols, new sectors and more civilians to service them... but the big one here is terran space. Lots of ships, lots of sectors. There's probably a lot of room for performance trimming in the jobs file.

Definitely Jobs is one of the first things to look at in terms of potential optimization.

Litcube · Post by **Litcube** » Thu, 30. Oct 08, 03:08

Maybe we should take positional snapshots of various regions in space where we've found the most significant slow downs and compare our statistics.

http://members.shaw.ca/litcube/X3/Bench ... fLight.zip

Run that script in Home of Light, stock game, and it will set your x,y,z alpha,beta,gamma to exactly what I saw. I can't break the 30 FPS barrier regardless of my graphics settings (No AA, No AF, LOW textures and shading, 1024 x 768). Totally embarrasing.

I'm not running any small rig either:

3.0 GHz CPU
2 GB Ram
Windows XP
GeForce 8800 GTX

Observe · Post by **Observe** » Thu, 30. Oct 08, 06:31

Good ideas Litcube. I haven't much time of late since I'm focusing on converting Transcend to TC. However, I am starting to make a few performance tests as I go along.

For example, I have an init script for startup in Moon sector. Game is otherwise not modified. With normal Jobs file in place I have around 35-48 FPS (measured with FRAPS 2.8.0). If I remove Jobs file, I have between 90-95 FPS. Clearly AI ship spawning is having a fairly significant impact.

I'll be digging into this in more detail as I optimize Transcend. In the meantime, hopefully ES is working on improvements.

My Graphics settings are default:
[ external image ]

Litcube · Post by **Litcube** » Thu, 30. Oct 08, 06:35

Observe wrote:If I remove Jobs file, I have between 90-95 FPS. Clearly AI ship spawning is having a fairly significant impact.

Wow! That's pretty clear evidence.

Not to be a negative nancy, but the only reason I stayed away from Xtended was because I found the scripts to be written by a plethora of people who all assumed that their scripts would be the only scripts running in the Universe.

Meaning, extremely taxing. I found a lack of performance optimization in many of the setups and global scripts.

Observe · Post by **Observe** » Thu, 30. Oct 08, 06:47

Litcube wrote:...Meaning, extremely taxing. I found a lack of performance optimization in many of the setups and global scripts.

I agree with you there Litcube - which is why my personal policy has always been to run the absolute minimum of scripts both in my own game and in the mods I develop.

Litcube · Post by **Litcube** » Thu, 30. Oct 08, 07:12

That's something I follow myself.

So, I think we can draw a preliminary hypothesis, accruing various information across the forums in regards to low CPU & GPU resource allocation while at the same time offering up sub 40s frame rates, and also with the addition of various jobs.txt minimizations (including yours), that the crux of problem likely lies in script optimization.

Here’s my theory: the game itself seems to govern its own speed in terms of "ticks" when it has to deal with script processes. Graphical frame rate update and script processes are probably married to one another in some sort of ratio. This is a good thing, actually, as it prevents parts of the game from chugging unnoticeably, while you’re blissfully unaware of that the de-synchronization is wreaking havoc on your game Universe’s economic state. As only one script process can run at a time, and we've seen it with faulty loops with no WAITs, you can get 0 FPS due to a script. Thusly, you could also be relegated to sub 10 FPS with a massive search script with sparse and intermittent WAITs. We’re getting slightly better than that in some situations, along with extensive stretches of global check scripts running without proper WAITs causing this stuttering that is widely reported.

I’ve watched the global script task list for a few minutes during times of trouble, and I saw various scripts pop up for a number of seconds in relation to restocking Terran stations with shields, and God knows what else. As an aside, why can’t Terran stations restock just like every other station in the economy? Why would it deserve special treatment?

I’ll dig.

Gazz · Post by **Gazz** » Thu, 30. Oct 08, 07:19

Litcube wrote:As an aside, why can’t Terran stations restock just like every other station in the economy? Why would it deserve special treatment?

Because the terran economy practically doesn't exist.

Sartorie · Post by **Sartorie** » Thu, 30. Oct 08, 07:30

my slowdowns are mostly a mix of rocks in a sector (home of light for example) and the total CPU stress by the ships in the universe - remember that each and every ship runs a script too, fires signals around etc.

what also helps is getting rid of civilians as I had up to 35 of these per sector which is a bit overkill for my personal taste

for comparison : my XTM game had about 7800 ships in total (vanilla should have way less), my TC game has 8300 ... which means TC is quite heavy on the CPU compared to X3 vanilla just in terms of ship numbers (these numbers to not include civilian ships)

p.s. tips for script optimisations are always welcome

Litcube · Post by **Litcube** » Thu, 30. Oct 08, 07:37

That's the problem, Sartorie. The CPU isn't the bottleneck, it's the game itself. There's been numerous studies, including my own, that show the CPU not being utilized as much as expected.

Consider: Even the fastest CPU known to man, 10 years from now even, would run at 0 FPS due to a script with no WAITs. It's how the game is designed. It's designed to *wait* for script processes to finish before updating anything else, including graphical frames.

I had no idea there were that many ships! Thanks for the tip!

@Observe: Just used my BenchMark.HomeOfLight after completely cleaning my script directory. Deleted all scripts.

I can now knock out 38 FPS with that scene.

Sartorie · Post by **Sartorie** » Thu, 30. Oct 08, 07:53

I know what you mean, the interesting question of single threaded, event-driven programming is when to have a program play nice with all the others e.g. run with small time slices alongside many others and when to just give it a big timeslice if it only runs for a (very) short time and is then removed from the scheduler

sometimes the overhead of playing nice is not worth paying - but most of the time it is

about the terrans ... for my taste they could be removed from the game as once you have seen them there is not much to do there - bit of a let down

PhantomDG1 · Post by **PhantomDG1** » Thu, 30. Oct 08, 16:19

I haven't encountered any performance problems as such ! Although any possibility of improved performance would be greatly appreciated !

My rig is somewhat older (although upgraded)

3.2 Ghz
Nvidia 9600 GT card
3 Gb RAM

(On a slightly different note I'd be interested to know what poly counts are used on some of the new station models as I'm currently working on some additional variations of my own and it would be nice to see if I can safely up my alloted poly count !)

In short... any improvement in perfomance... can't be a bad thing ! ! !

Phantom

egosoft.com

egosoft.com

[MOD IDEA] X3 Terran Conflict - improved performance

[MOD IDEA] X3 Terran Conflict - improved performance