[MOD] Asteroid Fade-in

The place to discuss scripting and game modifications for X4: Foundations.

Moderators: Moderators for English X Forum, Scripting / Modding Moderators

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

[MOD] Asteroid Fade-in

Post by SirNukes » Sat, 30. May 20, 21:23

Changes asteroids to fade in instead of pop-in-instantly.

Download "sn_asteroid_fade" as a zip from:
https://github.com/bvbohnen/x4-projects/releases

Egosoft has given reasons why they have asteroids pop in, but I didn't really like them (the reasons or the popping). So I fixed it. Took about 10 hours, mostly learning the X4 material system and graphics shaders and testing in-game through restarts. (More hours were needed to deal with xxl asteroid zone-based culling.)

Notes:
- XXL asteroid display distance increased out to 200km. (Vanilla pops them in groups where asteroids can vary from ~40km out to >100km in the group due to zone shenanigans).
- XXL asteroids cannot be targeted past 30km, similar to other asteroids. (This is a restriction of the engine, as targetable asteroids get culled when a couple zones away, eg. as little as ~40km.)
- Other asteroid render distances unchanged.
- Fading is done by dithering between the asteroid and its background, which may be noticeable in high-contrast situations (eg. a light colored skybox nebula behind the rock).
- FPS impact should be small, just a few extra large asteroids showing in the distance.
- For existing saves, existing XXL asteroids may continue to pop in at close distance; this may clear up over time.
- The extension size of ~30 MB is mostly due to binary edits of asteroid lod xmf files to uniquify their material references.
- Generated by an X4 Customizer script.
- May not work well with other mods that change asteroid render distances, since new material entries are generated based on vanilla distances. Some script tweaks may be needed to create adaptations for other mods.
Last edited by SirNukes on Wed, 3. Jun 20, 19:04, edited 3 times in total.

Max Bain
Posts: 1458
Joined: Wed, 27. Jun 18, 19:05
x3ap

Re: [MOD] Asteroid Fade-in

Post by Max Bain » Sat, 30. May 20, 21:53

Just tested it and man this is so much better - thank you.
I have increased the LOD distances even further in a separate mod and its still playable. With the mod that reduces fog density I think the average performance now is nearly as good as vanilla but with way longer view distances and no more popping asteroids :).

Anyone know what the worst system with many asteroids for performance is?
XR Ship Pack (adds several ships from XR) Link
Weapon Pack (adds several new weapons) Link
Economy Overhaul (expands the X4 economy with many new buildings) Link
X4 Editor (view stats of objects and make your own mod within a few clicks) Link

Buzz2005
Posts: 2184
Joined: Sat, 26. Feb 05, 01:47
x4

Re: [MOD] Asteroid Fade-in

Post by Buzz2005 » Sat, 30. May 20, 22:03

nopileus fortune or second contact, nopileus is the worst on my machine

the mod looks perfect, tried a matrix system full of roids, use maxs draw distance no problem fps or anything

thank you
Fixed ships getting spawned away from ship configuration menu at resupply ships from automatically getting deployables.

djrygar
Posts: 1841
Joined: Mon, 10. Aug 09, 02:09
x3ap

Re: [MOD] Asteroid Fade-in

Post by djrygar » Sat, 30. May 20, 22:37

yep, performance impact is acceptable.
Question tho, why dithering, not regular alpha? Some objects (like teladi trading station) do have smooth alpha blending, so there is mechanism in engine for that?
Kinda unpleasant shimmering is amplified by the fact that effect seems kind of attached to ship - dither mask moves with the ship when you stop and change angle. I know it has to be connected with camera, but if it has to be dither, maybe there is a way to keep it in place when camera/ship changes angle of view? (not distance). If this mask would stay in place, I think even dithering would not be so noticeable (esp in high res).

Looks much, much better anyway. This pop-in was horrible. Thanks for making it!

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [MOD] Asteroid Fade-in

Post by SirNukes » Sat, 30. May 20, 23:11

Alpha is harder to set up with the files as-is (eg. I didn't see how to do it easily), and more impactful as I understand it since objects have to be rendered on a different render pass or something, and apparently a 100% opaque object with alpha enabled still has a lot of overhead that non-alpha objects don't have.

The selection of which fragments to discard during dithering is difficult to get perfect (at my level of knowledge). I explored three pieces of information the fragment shader has: where the fragment is in the world, how far is it from the camera, and where it is on the screen. Currently I use screen position, which has the drawback of the dither pattern tracking camera rotation. But using fragment position or distance is even worse since it creates a shimmer effect as asteroids rotate.

Something better might be possible with more effort, eg. a texture mask mapped on top of the asteroid to define which points are shown at different dither ratios. But I decided the current version is good enough to release for now.

djrygar
Posts: 1841
Joined: Mon, 10. Aug 09, 02:09
x3ap

Re: [MOD] Asteroid Fade-in

Post by djrygar » Sat, 30. May 20, 23:24

well, it is good enough, especially when compared with vanilla offering ;)

I'm using DeadAir's Gate overhaul, which has enlarged sectors. With those short LODs it looked quite empty sometimes, so difference is pretty huge for me.

There was another thing that bothered be re; asteroids - when they change LOD, they change their displacement maps in kinda abrupt way. Like if those textures between LODs completely did not match. Do you know reason for that? It's unfixable by modding I guess, I am just curious why they did that.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [MOD] Asteroid Fade-in

Post by SirNukes » Sun, 31. May 20, 00:06

I haven't looked at lod level switching. Might be mitigated by increasing distances in renderparam_library or copying higher lod files over the lower lod ones, though with a possible fps impact. Redoing the lod models to match up better might be hard for a modder.

kmunoz
Posts: 444
Joined: Sun, 25. Feb 07, 19:59
x4

Re: [MOD] Asteroid Fade-in

Post by kmunoz » Sun, 31. May 20, 08:58

I really like the idea of the mod, and it looks good, but unfortunately it absolutely murders my frame rate. I normally get about 40fps in asteroid fields - with this mod it drops to under 20, sometimes under 10. I get that this isn't a "fault" of the mod - it's the nature of having more things on screen - I just wish I could use it!

(As a test, I went out into open space with the mod on, saved, removed the mod and restarted, then went back into the save. I went from 18 fps with the mod ON, to 41 fps with the mod OFF. There were only a few asteroids on screen in both cases.)
Let's Play Poorly! - Suboptimal X4 Playthroughs

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [MOD] Asteroid Fade-in

Post by SirNukes » Sun, 31. May 20, 09:07

Which system gives trouble?

Mycu
Posts: 408
Joined: Fri, 28. Sep 12, 08:30
x4

Re: [MOD] Asteroid Fade-in

Post by Mycu » Sun, 31. May 20, 11:35

I think that this one will get essential quickly.
Tried it in game - it improves immersion muuuch. No more poping asteroids.
In my opinion it's a significant improvement over vanilla approach.

-----------
And made a simple performance test in 1440p:
Argon Prime without mod: 77-84 FPS
Argon Prime with mod: 60-65 FPS
Last edited by Mycu on Sun, 31. May 20, 13:48, edited 1 time in total.
X3 AP 3.1 + XRM
X3 AP 3.1 + LU

Max Bain
Posts: 1458
Joined: Wed, 27. Jun 18, 19:05
x3ap

Re: [MOD] Asteroid Fade-in

Post by Max Bain » Sun, 31. May 20, 13:28

Btw, is there a mod that reduces the density of asteroid fields? Should be a good way to increase performance and there are more than enough asteroids for mining anyway.
XR Ship Pack (adds several ships from XR) Link
Weapon Pack (adds several new weapons) Link
Economy Overhaul (expands the X4 economy with many new buildings) Link
X4 Editor (view stats of objects and make your own mod within a few clicks) Link

kmunoz
Posts: 444
Joined: Sun, 25. Feb 07, 19:59
x4

Re: [MOD] Asteroid Fade-in

Post by kmunoz » Sun, 31. May 20, 17:17

SirNukes wrote:
Sun, 31. May 20, 09:07
Which system gives trouble?
I ran my test in Hatikvah's Choice I (which doesn't have an asteroid field, so I'm not sure what's going on there). I just re-tested to be sure I wasn't merely sleep deprived, and my save with the mod OFF was at 42-44 fps, and with the mod ON was 21-22 fps. My view was facing a single asteroid.

I noticed the problem in every other sector I went to, but I only tested in HAT 1. The slowdown was never permanently egregious - it got down to 10 fps for a few seconds but usually hovered around 18-22, where most of the time I'm 30-40 with dips down to 20 on occasion.

Now, a major caveat: I run a massive quantity of mods, so it's certainly possible there's an interaction that's causing the issue. A few of them are visuals (GemFX, STARS, fire and smoke, textures from Rebirth), but most are not. I'm not running VRO but I am running FE.

Mycu's test showing a drop of about 15-20 fps is consistent with my experience, the only difference being is that 15-20 fps is half my framerate, but only 25% of his.
Let's Play Poorly! - Suboptimal X4 Playthroughs

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [MOD] Asteroid Fade-in

Post by SirNukes » Sun, 31. May 20, 20:02

After some testing, it looks like the defaults.xml zone visibility range is used to control:
- When large asteroids pop-in, since they are oddly tied to zones instead of region_lodvalues like other/smaller rocks.
- When ai scripts switch between high attention and low attention logic (and maybe similarly for ship movement accuracy).

So that would explain the fps hit in sectors without asteroids, since I upped the zone visibility range to prevent large asteroids popping or fading in too close. Modded games also tend to have more ships flying around in each sector.

I will try to find a solution to this problem, but nothing uncompromising comes to mind. It's unfortunate that graphics and ai are tied together like this.

Max Bain
Posts: 1458
Joined: Wed, 27. Jun 18, 19:05
x3ap

Re: [MOD] Asteroid Fade-in

Post by Max Bain » Sun, 31. May 20, 22:38

SirNukes wrote:
Sun, 31. May 20, 20:02
After some testing, it looks like the defaults.xml zone visibility range is used to control:
- When large asteroids pop-in, since they are oddly tied to zones instead of region_lodvalues like other/smaller rocks.
Are you sure about that? Because I have a private mod that drastically increases visibility range for asteroids without changing anything in the defaults.xml file. I only changed region_lodvalues.xml for this. Or am misunderstanding you?
XR Ship Pack (adds several ships from XR) Link
Weapon Pack (adds several new weapons) Link
Economy Overhaul (expands the X4 economy with many new buildings) Link
X4 Editor (view stats of objects and make your own mod within a few clicks) Link

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [MOD] Asteroid Fade-in

Post by SirNukes » Sun, 31. May 20, 23:50

Max Bain wrote:
Sun, 31. May 20, 22:38
Are you sure about that? Because I have a private mod that drastically increases visibility range for asteroids without changing anything in the defaults.xml file. I only changed region_lodvalues.xml for this. Or am misunderstanding you?
Try out the big ones, those that show up far away on the map after you long range scan and then switch to standard scan mode. Target one among a cluster, and fly toward it. The cluster will all pop in around 50-100km out, even though the lodvalues has them set to ~250km. Further away smaller rocks will already be showing before the big ones pop in.

ziplock815
Posts: 296
Joined: Wed, 8. Oct 14, 20:30

Re: [MOD] Asteroid Fade-in

Post by ziplock815 » Mon, 1. Jun 20, 00:04

Max Bain wrote:
Sun, 31. May 20, 13:28
Btw, is there a mod that reduces the density of asteroid fields? Should be a good way to increase performance and there are more than enough asteroids for mining anyway.
Would love to use this one really. It's sad to see how fps drops to 10 in some sectors like Second Contact and Nopileo's Fortune.

Falcrack
Posts: 4929
Joined: Wed, 29. Jul 09, 00:46
x4

Re: [MOD] Asteroid Fade-in

Post by Falcrack » Mon, 1. Jun 20, 05:52

I would love to use this mod if it did not have an FPS impact. But seeing as how the fps impact appears significant, I'll take higher FPS over popping in asteroids!

Max Bain
Posts: 1458
Joined: Wed, 27. Jun 18, 19:05
x3ap

Re: [MOD] Asteroid Fade-in

Post by Max Bain » Mon, 1. Jun 20, 11:45

@SirNukes:

I just had a look into your mod. I tried to lower the distance in defaults.xml but then I had pink steroids (only the very big ones). As if they had no texture at all, no matter how close I flew. Why is that? What value determines when the textures will be loaded correctly?
edit:
Ok it seems they only appear when I teleport into an asteroid field and then all big chunks in a specific radius are pink.

edit2:
I tried to lower the "visible distance" in defaults to test how it will look if asteroids fade in closer but even 200.000 will remove your fade in effect completely for all asteroids. Is there a value I can adjust to say the game when asteroids shall start to fade in? I cant find it so far.
XR Ship Pack (adds several ships from XR) Link
Weapon Pack (adds several new weapons) Link
Economy Overhaul (expands the X4 economy with many new buildings) Link
X4 Editor (view stats of objects and make your own mod within a few clicks) Link

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [MOD] Asteroid Fade-in

Post by SirNukes » Tue, 2. Jun 20, 09:05

Version 1.1 is up, with an alternate method of keeping XXL asteroids drawing that doesn't put aiscript into high attention mode.

This actually took a while to figure out. In the end, turns out that an asteroid that is targetable has an active component, and when it has a component it gets culled from vision if its zone isn't "visible". The solution is to reduce targeting distance on XXL asteroids down to 30-40km or so. A couple quirks to this: existing saves still have the long distance components on their xxl asteroids, so they can still get some pop-up. I don't know if that will fix itself over time. Secondly, flying away from a targeted asteroid will keep its component alive, which will cause it to vanish when its zone goes low attention; the asteroid will reappear when clearing the target if still inside the fade distance. (Those two quirks really threw off my testing.)

@Max Bain
Certain files need to be in a subst cat/dat to work. It sounds like you maybe extracted the shader spec file? I recall that causing purple textures. Anyway, default zone visibility range is no longer modified in 1.1.

The fade distance is set by custom asteroid lod xmf files tying into uniquified per-asteroid material entries which define start/end fade distances that match the region_lodvalues. They then link to uniquified shaders that are modified to enact the dithering.

Mycu
Posts: 408
Joined: Fri, 28. Sep 12, 08:30
x4

Re: [MOD] Asteroid Fade-in

Post by Mycu » Tue, 2. Jun 20, 13:50

Tried version 1.1 few minutes ago. Improvement in FPS is significant.
Currently (on the same save that I've checked previusly) it seems that I have same FPS with and without mod.
Only noticable thing is vanilla LOD change behaviuor.
Nice improvement.
X3 AP 3.1 + XRM
X3 AP 3.1 + LU

Post Reply

Return to “X4: Foundations - Scripts and Modding”