Homework Assignment for Egosoft

[Falcrack]

Here's the assignment. Create a large mixed fleet of destroyers, fighters, etc., move them a couple of sectors over to a Xenon sector, and while in the sector yourself, have a grand battle with a Xenon fleet, and attempt to have fun. Take off the bug fixing developer hat and just try to be a player who is looking to have a good time.

Here are some things I would try to do to make large scale fleet combat more fun.

-Find ways to increase the ability of your fleet to move together, cohesively
-Find ways to increase the transit speed (ie ability to use travel drive of ships in a fleet)
-Address the behavior where ships will fly to the last known location of the enemy approximately 1-2 minutes ago, instead of flying to where is enemy is right now
-Find ways to make the fleet become more responsive and easy to use when fighting
-Observe any "dumb" combat behaviors and think of how this behavior might be improved
-Most importantly, for any other aspects of fleet combat which make it feel "unfun", cumbersome, and generally buggy, come up with ways that would make it more fun.

Large scale fleet combat in X4 is severely lacking in the fun department. Improve it, and you will win over many of the people who left negative reviews on Steam. No amount of DLC or extra ships will help if the "foundation" for combat is as lacking as it is right now.

[Imperial Good]

-Address the behavior where ships will fly to the last known location of the enemy approximately 1-2 minutes ago, instead of flying to where is enemy is right now

This happens because it cannot keep updating the target position constantly, likely to keep CPU overhead down, and instead only does when it approaches where the ship last was.

The actual solution would be to have the ship guess where the target will be by the time it catches up and so fly an intercept course. The estimate does not need to be perfect, it just has to be good enough that it stands a reasonable chance of arriving within firing distance as opposed to currently where slower ships such as frigates might never be able to intercept a L Destroyer because they are constantly playing catch up with it.

If the intercept time by flying is reasonable (<20 seconds) then just fly into battle. Otherwise travel mode. If the intercept time is as good as impossible (target flying faster than ship travel speed so no future point of interception, or intercept is over 90 seconds) then resort to current behaviour and try to intercept again once arrived.

[Kadatherion]

-Address the behavior where ships will fly to the last known location of the enemy approximately 1-2 minutes ago, instead of flying to where is enemy is right now

This happens because it cannot keep updating the target position constantly, likely to keep CPU overhead down, and instead only does when it approaches where the ship last was.

Yeah, this exactly what happens for *every* follow task in the game (which is part of the attack routine: follow the target to enter attack range). Indeed the reason why it's like that must be to save CPU cycles, but there's a pretty big but: it wasn't like this in previous games, it updated pretty much real time. Granted it's not the same game anymore, so it's relatively a moot point to say "but in X3 that wasn't an issue!", but still if you have to shave off some feature to keep the CPU demand in check, this is the last thing you should butcher, as it's absolutely fundamental to making everything else working decently and being fun.

As the follow routine is such a common component of many of the tasks every AI ship is doing around the universe I'm sure gutting it as they did is a large part of the reason why the game is relatively fine performance wise (except in certain situations, like too close to certain stations or on the map), but there's little point in smooth gameplay if the gameplay itself is then broken AF. Your suggestion of having the script estimate the position of the target ship given how long it would take to intercept it would still be less than optimal (wouldn't work for instance when the follow command had yourself as the target, as it couldn't predict anything), but it would already be a much better compromise at least in some pretty important chunks of the game.

[graphicboy]

Wait... you tried to do a fleet battle in sector? Haha.

Really? (/sarc)

Fear not, your fleet chases empty space OOS also.

[Faustov]

-Address the behavior where ships will fly to the last known location of the enemy approximately 1-2 minutes ago, instead of flying to where is enemy is right now

This happens because it cannot keep updating the target position constantly, likely to keep CPU overhead down, and instead only does when it approaches where the ship last was.

The actual solution would be to have the ship guess where the target will be by the time it catches up and so fly an intercept course. The estimate does not need to be perfect, it just has to be good enough that it stands a reasonable chance of arriving within firing distance as opposed to currently where slower ships such as frigates might never be able to intercept a L Destroyer because they are constantly playing catch up with it.

If the intercept time by flying is reasonable (<20 seconds) then just fly into battle. Otherwise travel mode. If the intercept time is as good as impossible (target flying faster than ship travel speed so no future point of interception, or intercept is over 90 seconds) then resort to current behaviour and try to intercept again once arrived.

I am surprised by your definition of the problem and solution here. This is a simple issue to be resolved by interpolating the movement vector from 2 points of the visible ship path, possibly repeated over time. There cannot be even a moderate concern about the CPU here unless the developers are doing something seriously wrong.

[Kadatherion]

I dare say this is way too heavy handed for no good enough reason anyway. I'm fairly sure if the course were to be recalculated every few seconds (not milliseconds, even 5-10 full seconds) the CPU impact would still be relatively insignificant, and that would already be more than enough to avoid most issues, and the not absolutely "real time" update rate still wouldn't be much noticeable unless you were sitting there looking just to point your finger at that.

[AleksMain]

I dare say this is way too heavy handed for no good enough reason anyway. I'm fairly sure if the course were to be recalculated every few seconds (not milliseconds, even 5-10 full seconds) the CPU impact would still be relatively insignificant, and that would already be more than enough to avoid most issues, and the not absolutely "real time" update rate still wouldn't be much noticeable unless you were sitting there looking just to point your finger at that.

I am sorry to disappoint you, but please, read following post:

Yes, a path-finding is not an easy problem, especially in 3D, but I was watching CPU usage for X3 game, and it's obvious it's not the (main) problem. The pathfinding simply hasn't been done properly, period. Terran sectors were a story for itself...

There wasn't any path-finding in X3, period. Ships flew in a straight line towards their target. If something got in the way they'd fly away from it for a bit then resume flying towards their target. Anything more than that was impractical in a single-threaded game.

As various people have already pointed out, the problem with path-finding is not just that it's "hard", it's that it's expensive in CPU time. Remember that this is not a "normal" game which only has to deal with the objects in a particular area; everything, everywhere is being simulated. Some shortcuts can be taken when the player isn't around, but even that doesn't solve everything. Threading also helps, but again it's not a magic bullet.

[Falcrack]

Perhaps they could keep the slow update for pathfinding when the target is static, like stations and waypoints. When the waypoint changes, it gets an update in pathfinding, and when the target is mobile, it gets more frequent (maybe every 2 seconds) updates.

I find it frustrating when the behavior of ships attacking other ships is to fly to the last known location, then slowly slow down until stopped, not responding to anything regardless of whether they come under sudden attack during their course inbound to where the target was.

I should also add, that they should consider improving the visuals of combat, especially capital ship combat. We need big explosions that fill half the sky with bright shockwaves when a destroyer explodes, fires on damaged capital ships, more colorful shots, higher speed plasma bolts. It's been raised as an issue before, but hopefully Egosoft don't simply forget about this issue as well.

[Nort The Fragrent]

I want Synchronised swimming as a fleet manoeuvre ! Have the ships turn upside down twiddling the turrets above water so to speak that would confuse the enemy.

Or perhaps the cross pincer like a pair of badly made pliers with a loose pin !

Oh! Gosh ! You lot not waring again, go into the fight, get your noses broken. Limp home a complain you lost, or won, whatever.

Stop giving the Developers a hard time, they need to fix the paint on our ships first then there are the curtains that need replacing. And the new sofa to carry up those stairs to the new to be introduced cockpit of the coffee ship.

[Imperial Good]

I am surprised by your definition of the problem and solution here. This is a simple issue to be resolved by interpolating the movement vector from 2 points of the visible ship path, possibly repeated over time. There cannot be even a moderate concern about the CPU here unless the developers are doing something seriously wrong.

Except ships in low attention do not get updated every frame. Their logic runs sporadically or periodically. Yes for 1 ship it is not a CPU concern, but for thousands, which there are, it is.

Hence the solution is to try to intercept the target rather than try to fly to where it was at the point in time the flight was calculated.

I'm fairly sure if the course were to be recalculated every few seconds (not milliseconds, even 5-10 full seconds) the CPU impact would still be relatively insignificant, and that would already be more than enough to avoid most issues, and the not absolutely "real time" update rate still wouldn't be much noticeable unless you were sitting there looking just to point your finger at that.

This is what it does, except in the case of slower craft it causes issues such as Ospreys never being able to catch up with a target Behemoth Vanguard. This is because they travel mode until nearby where the target was then go back to flying, then recalculate and then travel mode, etc. If they travel mode flight in front of the target then there would be no problem.

[Faustov]

I am surprised by your definition of the problem and solution here. This is a simple issue to be resolved by interpolating the movement vector from 2 points of the visible ship path, possibly repeated over time. There cannot be even a moderate concern about the CPU here unless the developers are doing something seriously wrong.

Except ships in low attention do not get updated every frame. Their logic runs sporadically or periodically. Yes for 1 ship it is not a CPU concern, but for thousands, which there are, it is.

Hence the solution is to try to intercept the target rather than try to fly to where it was at the point in time the flight was calculated.

Actually my jaw dropped after reading CBJ's comment that this CPU concern is real. Remember 1998, where you could launch 20 bots playing well against you on a super fast aerial q3dm17 map? You could still get 60 fps on the hardware you got back then. Hope this is enough for a comment.

[burger1]

Use carriers to move fleets. They can dock 10 m ships and 40 s ships. Instead of moving a fleet of ships you can just move one ship. I think the slowest carrier with the slowest engines can move at at least 230 m/s when fully modded.

It looks like most ships auto dock/launch in the latest patch. You can also just set ships to auto dock at a certain ship so that after a set of commands is completed they auto dock at that ship.

Use s/m ships that can travel faster than your carrier.

Destroyers are more for light sector patrol, support and destroying stations.

[lordmuck]

Sorry but I think in this day and age with this hardware that the whole "its too much for the cpu to handle " is bull.. And also if that is indeed the case then clearly it has been coded absolutely poorly. The whole fly to a target (not its last stupid position f'ing everything up) has been done many times and works with no issues on other titles, but only here it does not and people think that its ok and correct due to the game/engine/everything else that the game is doing.. unbelievable

Edit: Yes Egosoft please go play your game as mentioned by the OP..

[Imperial Good]

Sorry but I think in this day and age with this hardware that the whole "its too much for the cpu to handle " is bull.. And also if that is indeed the case then clearly it has been coded absolutely poorly. The whole fly to a target (not its last stupid position f'ing everything up) has been done many times and works with no issues on other titles, but only here it does not and people think that its ok and correct due to the game/engine/everything else that the game is doing.. unbelievable

Edit: Yes Egosoft please go play your game as mentioned by the OP..

Modern CPUs have a per thread performance not much faster than those a decade ago. They have more cores yes, but using more cores is difficult and not always possible for all applications. Even AAA games from EA, Ubisoft, Activision, Square Enix, etc struggle to do so efficiently and they have budgets orders of magnitude larger than Egosoft.

Yes I agree X4 can be improved, practically everything can if enough money is thrown at it, but that is not always an option. As it is a large number of topics state how X4 has performance issues.

[sh1pman]

Sorry but I think in this day and age with this hardware that the whole "its too much for the cpu to handle " is bull.. And also if that is indeed the case then clearly it has been coded absolutely poorly. The whole fly to a target (not its last stupid position f'ing everything up) has been done many times and works with no issues on other titles, but only here it does not and people think that its ok and correct due to the game/engine/everything else that the game is doing.. unbelievable

Edit: Yes Egosoft please go play your game as mentioned by the OP..

Modern CPUs have a per thread performance not much faster than those a decade ago. They have more cores yes, but using more cores is difficult and not always possible for all applications. Even AAA games from EA, Ubisoft, Activision, Square Enix, etc struggle to do so efficiently and they have budgets orders of magnitude larger than Egosoft.

Yes I agree X4 can be improved, practically everything can if enough money is thrown at it, but that is not always an option. As it is a large number of topics state how X4 has performance issues.

If this is the case, then it seems that Egosoft bit more than they could chew. They made a game that fundamentally can’t run well on modern consumer hardware. It’s either performance issues or gameplay issues, choose one. Well done I guess.

[Nort The Fragrent]

Throwing money at a problem seldom works, It’ the same as throwing more labour into a house build. All stepping on each other and nothing gets done.
What is needed is just one clever programmer, who knows what they are doing. And that is an issue in so many professions. Too many people who think they know what they are doing… As they have a piece of paper they got whilst on holiday at University.
Employ the right person for the job, not your mate.

CPU, functionality is clear fast and if programmed correctly will throw out FPS with no problem. It’s the bad programming that is the problem. The human element, Not the hardware!
Presenting this FPS to an (AI program), is probably the way to get better FPS. We humans are stupid, old fashioned, and riddled with pride. Thats the problem…..

[Imperial Good]

What is needed is just one clever programmer, who knows what they are doing. And that is an issue in so many professions. Too many people who think they know what they are doing… As they have a piece of paper they got whilst on holiday at University.
Employ the right person for the job, not your mate.

The issue is still that of an economical one. Since that same clever programmer could spend his time fixing up the combat system of this game, or instead fixing up the trade AI, or instead optimizing the physics engine, or instead adding new features such as for sector control. Yes they could do all of them, but that might take years. Games with larger budgets get around this by throwing more programmers at the problem, which is not really an option for smaller development teams.

[Faustov]

Throwing money at a problem seldom works, It’ the same as throwing more labour into a house build. All stepping on each other and nothing gets done.
What is needed is just one clever programmer, who knows what they are doing. And that is an issue in so many professions. Too many people who think they know what they are doing… As they have a piece of paper they got whilst on holiday at University.
Employ the right person for the job, not your mate.

CPU, functionality is clear fast and if programmed correctly will throw out FPS with no problem. It’s the bad programming that is the problem. The human element, Not the hardware!
Presenting this FPS to an (AI program), is probably the way to get better FPS. We humans are stupid, old fashioned, and riddled with pride. Thats the problem…..

I think this sums it up well.

[Kadatherion]

This is what it does, except in the case of slower craft it causes issues such as Ospreys never being able to catch up with a target Behemoth Vanguard. This is because they travel mode until nearby where the target was then go back to flying, then recalculate and then travel mode, etc. If they travel mode flight in front of the target then there would be no problem.

Except it's not what (at least often) happens. Order a far away ship to follow YOU. It will fly to where you were at the moment you issued the order. If you then fly somewhere else, it will not lag behind a few seconds: it will fly to those coordinates completely ignoring any recalculation until it gets there. Only then, if it doesn't find you, it will look where you are now and update the coordinates. This means they can lag behind entire *minutes*, as by when the ship arrives you could be in the opposite corner of the sector. The follow command/task - at least when you are the target - doesn't get slow updates while en route: it gets no updates at all.

[graphicboy]

Except it's not what (at least often) happens. Order a far away ship to follow YOU. It will fly to where you were at the moment you issued the order. If you then fly somewhere else, it will not lag behind a few seconds: it will fly to those coordinates completely ignoring any recalculation until it gets there. Only then, if it doesn't find you, it will look where you are now and update the coordinates. This means they can lag behind entire *minutes*, as by when the ship arrives you could be in the opposite corner of the sector. The follow command/task - at least when you are the target - doesn't get slow updates while en route: it gets no updates at all.

I stumbled into a TODO around this in 08.dat, but can't find it now. I want to think there's some aspect of the flight system that isn't done yet. *shrug* Search 08 for TODO. It's enlightening.