[BUG] Any version - Planets slow to render when entering sector - Much improved in 4.10 b1.

Ask here if you experience technical problems with X4: Foundations.

Moderator: Moderators for English X Forum

User avatar
alt3rn1ty
Posts: 2298
Joined: Thu, 26. Jan 06, 19:45
x4

[BUG] Any version - Planets slow to render when entering sector - Much improved in 4.10 b1.

Post by alt3rn1ty » Fri, 8. Mar 19, 01:07

This has been a bug since day 1 release of X4 for me - Haven't seen it mentioned, and with currrent beta 2.20 installed it is still happening.

When you go through a jump gate, and the moment you arrive in the next sector, if you can see a planet it takes a couple of seconds to catch up and display its textures and atmosphere correctly ..

.. When you first see it there is a black hole surrounded by a white thick ring like a big Polo mint - Then the game rendering catches up and displays the planet as it should be.

Go to Company regard, then go through the jump gate to Hewa's twin II, you will see what I mean when you enter Hewa's twin II. Thats just one example, I think it happens all the time so long as you can see a planet when you arrive.

I have been keeping up to date with NVidia drivers (not betas), I think those have been updated 3 times so far, currently on v419.17
Laptop Dell G15 5510 : Win 11 x64
CPU - 10th Gen' Core I7 10870H 2.2-5.0ghz, GPU - NVidia Geforce RTX 3060, VRAM 6gb GDDR5,
RAM - 32gb (2x16gb, Dual Channel mode set in BIOS) DDR4 2933mhz Kingston Fury Impact,
SSD - Kioxia M.2 NVME 512gb (System), + Samsung M.2 NVME 970 Evo Plus 1tb (Games)

:boron: Long live Queen Polypheides and may her tentacles always be supple.
Seeker of Sohnen.

CBJ
EGOSOFT
EGOSOFT
Posts: 51741
Joined: Tue, 29. Apr 03, 00:56
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by CBJ » Fri, 8. Mar 19, 09:40

The game does try to pre-load them shortly before you enter the sector, but there can be a lot to load and sometimes it can't get everything loaded in time. This is one of those things that should improve if you install the game on a faster disk such as an SSD. Even on an HDD you may be able to improve it somewhat by setting the game folder to be excluded from your AV software's checks, as this can sometimes slow down file access.

Imperial Good
Moderator (English)
Moderator (English)
Posts: 4750
Joined: Fri, 21. Dec 18, 18:23
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Imperial Good » Fri, 8. Mar 19, 10:13

More system memory can help combat the frequency of the issue if running from mechanical drives. Windows caches recently accessed file pages in free memory so any subsequent access is performed extremely efficiently (faster than SSD). This will not help the initial load in as that is bottlenecked by the mechanical drive but should speed up subsequent load in as long as there is enough memory available that Windows does not need to evict game data files from the file cache. I am unsure of how Linux handles file IO but I would hope the same principles apply. This would be most relevant if running on 4, 6 or 8GB of main memory, since expanding to something closer to 12, 16 or 18 GB would potentially yeild improvements by greatly expanding the amount of memory available for the file cache while playing.

EDIT upon using RAMMap it appears that the way X4 does IO for game data does not allow Windows to cache game data file pages. As such more memory itself will not improve IO performance. Strangely it does allow save files to be cached as well as miscellaneous other files.
Last edited by Imperial Good on Sat, 9. Mar 19, 16:51, edited 1 time in total.

User avatar
alt3rn1ty
Posts: 2298
Joined: Thu, 26. Jan 06, 19:45
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by alt3rn1ty » Fri, 8. Mar 19, 10:38

The game does try to pre-load them shortly before you enter the sector
Does that also work when you enter a gate on a fast highway? (IE, is there a place on a highway that marks where the pre-loading should start, which would be a greater distance from the gate than if there was no highway, due to the acceleration and resulting shortened time it will take you to hit the gate) ?

I think I would prefer a blank screen delay until everything is ready to render correctly, rather than obvious game hiccups.
At least with a blank screen delay you can imagine / role play its a warping issue that is natural and to be expected.


The game is installed on an SSD. The SSD the game is installed on is an extra internal drive, and is not being used as my system drive (No Page file)

AV is windows Defender, set to ignore the X4 exe and the whole X4 Foundations folder aswell

Exclusions screenshot - https://i.imgur.com/4gIT3hB.png

This is something I routinely do, and has been the case since first installing the game, before I started my first game of X4. Its just one of the checkoffs to do like is the NVidia Driver profile for this game set to always use the High Performance Graphics Adaptor instead of the Intel Integrated one which lives on the Core I7 CPU as an auxiliary chip.

MSI Gaming laptop
System has Core I7, 2.8ghz - 3.4ghz (3.4ghz in boost mode), 4 cores
16gb System RAM - DDR3L 1600mhz
NVidia Geforce GTX 970
3gb VRAM - GDDR5


In the MSI Gaming Center App, the Device setting is set to High Performance, and Sports Mode is on.

Game settings - Since first release of the game, I recognise that my system is recommended for CPU having a good I7, but is close to minimum with the NVidia 970. The game by default sets everything to high for my machine which I think is a bit optimistic - So I do turn things down a bit, and have never increased any settings. So for instance the Textures are down at Medium, Shadows and SSAO etc I turn off because these games have always been CPU intensive ( I know my CPU is more than capable, but even then when an X game goes into a huge battle you need everything you can get, so I give it more resources to play with by turning down the fluff settings ).
I also even set the config.xml to <trafficdensity>0.40</trafficdensity> ( This is just an aesthetic thing for me, the default 0.50 has too much traffic around stations, but I suspect when things get busy turning this down is potentially going to help a bit )

I have no concerns about CPU or GPU temperatures, the game and machine otherwise perform very nicely, no fear of melting any solder with this game :P
Laptop Dell G15 5510 : Win 11 x64
CPU - 10th Gen' Core I7 10870H 2.2-5.0ghz, GPU - NVidia Geforce RTX 3060, VRAM 6gb GDDR5,
RAM - 32gb (2x16gb, Dual Channel mode set in BIOS) DDR4 2933mhz Kingston Fury Impact,
SSD - Kioxia M.2 NVME 512gb (System), + Samsung M.2 NVME 970 Evo Plus 1tb (Games)

:boron: Long live Queen Polypheides and may her tentacles always be supple.
Seeker of Sohnen.

User avatar
alt3rn1ty
Posts: 2298
Joined: Thu, 26. Jan 06, 19:45
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by alt3rn1ty » Wed, 8. May 19, 00:21

Still experience this issue in 2.50 beta 4
Laptop Dell G15 5510 : Win 11 x64
CPU - 10th Gen' Core I7 10870H 2.2-5.0ghz, GPU - NVidia Geforce RTX 3060, VRAM 6gb GDDR5,
RAM - 32gb (2x16gb, Dual Channel mode set in BIOS) DDR4 2933mhz Kingston Fury Impact,
SSD - Kioxia M.2 NVME 512gb (System), + Samsung M.2 NVME 970 Evo Plus 1tb (Games)

:boron: Long live Queen Polypheides and may her tentacles always be supple.
Seeker of Sohnen.

Kernel Panic
Posts: 86
Joined: Tue, 2. May 06, 15:18
x3tc

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Kernel Panic » Wed, 8. May 19, 05:38

Just to add my two cents...

I have the game installed to a Linux software RAID0 (mdadm) of 5 Toshiba 1TB 7200 rpm HDDs on a system with 16GB of DDR3 1600. I get sequential read and write speeds of over 800MB/s and while it is very rare sometimes it does take a second or so for the planet textures to load. At first I only saw it when I entered a really busy sector but as my game universe matures it is happening more often. It doesn't bother me nearly as much as the when the radar only shows squares of colors when entering a sector because knowing a planets texture will not save my life. Knowing if the red square in my radar is an "N" or "I" does make it easier to live.

Imperial Good
Moderator (English)
Moderator (English)
Posts: 4750
Joined: Fri, 21. Dec 18, 18:23
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Imperial Good » Wed, 8. May 19, 08:50

Kernel Panic wrote:
Wed, 8. May 19, 05:38
I have the game installed to a Linux software RAID0 (mdadm) of 5 Toshiba 1TB 7200 rpm HDDs on a system with 16GB of DDR3 1600. I get sequential read and write speeds of over 800MB/s and while it is very rare sometimes it does take a second or so for the planet textures to load. At first I only saw it when I entered a really busy sector but as my game universe matures it is happening more often.
800MB/sec sequential is meaningless for games which are all about random access performance as they require loading a lot of small pieces of data scattered around archive files very fast. Mechanical drives just suck for random access for physical reasons, and no amount of RAID can compensate unfortunately. The load performance you are getting is likely very close to that of just a single drive since ultimately the bottleneck will be how fast the platter and head can move. A single basic AHCI SSD from a reputable brand will have that configuration beat considerably in loading time at a fraction of the cost despite having significantly slower sequential read speed purely because its random access performance is orders of magnitude better.

Loading in X4 is primarily optimized for SSDs. It purposely bypass the OS file cache with the reason given by Egosoft that the game caches assets internally so there is no need for such double caching. This is in theory correct as double caching would degrade the performance of each other, although not what other game developers like Blizzard Entertainment believe. With the file cache and a decent amount of memory there are some tricks/hacks one can perform to improve load times on mechanical drives by using their sequential read speed to cache large pieces of data into the memory based file cache for quick access. The delay in asset pop in is significantly reduced when running off a SSD and even frame rate consistency is improved, likely as the result of fewer IO stalls.

There are two major bottlenecks when transitioning sectors. First is the random access performance needed to load the required game assets. To minimize this I hope (difficult to tell from a user perspective) that X4 performs reads for multiple required assets in parallel (queue depth >1) so as to achieve optimum random access read performance on SSDs, however this will not improve mechanical drive performance anywhere near as much. The other is the processing of the loaded assets, which at the very least require to be decompressed. Again this bottleneck can be minimized by loading multiple assets in parallel since that will take advantage of the multi core/thread performance of modern x86-64 processors. Now in theory IO should be the main bottleneck, however if they add in stages such as decryption, verification or even procedural variance generation that could push the bottleneck to the CPU.
Kernel Panic wrote:
Wed, 8. May 19, 05:38
It doesn't bother me nearly as much as the when the radar only shows squares of colors when entering a sector because knowing a planets texture will not save my life. Knowing if the red square in my radar is an "N" or "I" does make it easier to live.
You can hit pause/break when entering a sector to pause the game. The loading will continue while paused and eventually the icons should be loaded at which point you can unpause as you can see what is happening. This is effectively a manually controlled loading screen.

Back when I was running of a mechanical drive this was the only way one could play. Changing sectors resulted in such low frame rate and assets took so long to load in that for busy sectors one could not really do anything for 15+ seconds. Since moving to a SSD this is no longer as much of an issue, with pretty much everything being loaded within 5 seconds or so of entering even a very busy sector. The radar icons usually take ~1 second to load in. The system I run is heavily bottlenecked (SATA 2.0, Intel I7 920, below minimum requirements) so on a more modern system (M.2 NVMe, 4 GHz processor with better instructions per cycle) it should be significantly faster still.

That said I would suggest a "transition loading screens" setting in the options. This would be very useful for running X4 on mechanical drives, especially slower ones, since it would result in a smoother experience when transitioning between systems or teleporting around at the cost of loading screens.

User avatar
alt3rn1ty
Posts: 2298
Joined: Thu, 26. Jan 06, 19:45
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by alt3rn1ty » Wed, 8. May 19, 12:03

Imperial Good wrote:
Wed, 8. May 19, 08:50
~Snip~
You know when you buy X4, and you look at the System Requirements, it says nothing about what type of Hard Drive you need. Just 12gb space required.

I have SSD, but I could just aswell have had an internal laptop mechanical 3.5" IDE, and would still reasonably expect the game to work without issues related to the drive.

All I want to know is will Egosoft fix whatever the issue is.

Off topic slightly :

If you compare this game with something like Skyrim Special Edition, which has layers of textures applied to every object and even various parts (Head, Body, Hands, all different clothing and armour) of NPCs in game (Diffuse maps, Tangent Space Normal maps, Model Space Normal Maps, Specular maps, Glow maps, Reflectivity Maps, Environment Maps and Tone Maps) .. All of which have to be loaded from drive and decompressed from within an archive (BSA, Bethesda Softworks Archive).
The Skyrim game as a whole, has circa 33 thousand textures (thats not BS, I know this for a fact having reduced the majority of them for an old Skyrim texture mod I did, so that my previous aging laptop could play the game with its limited VRAM capacity)

But the speed of throughput is phenomenal, so much so that if you use a monitor of the Disk IO, and graph the read access, the game when it is loading creates a lot of caching of textures, but only needs up to 20% of disk IO access for that moment .. before the game has even loaded

During the game Disk IO rarely goes above 5% - Have a look at the first image here, top graph, purple trace line. During the game hard drive access is minimal, and thats for a game with a hell of a lot more going on in the resource loading department than X4 does.

X4 has Diffuse Maps, and Normal Maps, paint jobs and effects applied in real time. Many of which are re-used, and you have no landscape in space to worry about in terms of resources, maybe one nebulae overlay, maybe a couple of fog / star / effects textures ..

.. Point being, never seen any issues loading textures in Skyrim Special Edition which has a vast amount of resources to juggle from HD into the VRAM space available on your graphics card. X4's needs in this regard are far less, and should be able to pre-cache a planet texture (even if you hit the Gate at speed on a highway) and load it instantly.

As I mentioned to CBJ earlier, I think the point at which the game starts pre-caching the next sector probably needs to be given a bit more of a run up, more distance to the gate if you are on a highway approaching the gate, as opposed to just flying into the gate normally, because when you are on a highway you hit the gate and close the distance faster than you do if not on a highway, and so the pre-caching of resources for the upcoming sector you jump to has less time to do the necessary pre-loading. Thats my theory anyway, and CBJ did neither confirm or deny .. Maybe its an embarrassing oversight for the programmers?
Laptop Dell G15 5510 : Win 11 x64
CPU - 10th Gen' Core I7 10870H 2.2-5.0ghz, GPU - NVidia Geforce RTX 3060, VRAM 6gb GDDR5,
RAM - 32gb (2x16gb, Dual Channel mode set in BIOS) DDR4 2933mhz Kingston Fury Impact,
SSD - Kioxia M.2 NVME 512gb (System), + Samsung M.2 NVME 970 Evo Plus 1tb (Games)

:boron: Long live Queen Polypheides and may her tentacles always be supple.
Seeker of Sohnen.

Imperial Good
Moderator (English)
Moderator (English)
Posts: 4750
Joined: Fri, 21. Dec 18, 18:23
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Imperial Good » Wed, 8. May 19, 16:11

alt3rn1ty wrote:
Wed, 8. May 19, 12:03
You know when you buy X4, and you look at the System Requirements, it says nothing about what type of Hard Drive you need. Just 12gb space required.
I agree that looks like an oversight. The recommended specifications should explicitly state "SSD" since for a good experience without third party cache solutions one is needed. Minimum requirements it does not matter as I played for many months on a mechanical drive, it is not an ideal experience but still playable if you want to play.
alt3rn1ty wrote:
Wed, 8. May 19, 12:03
All I want to know is will Egosoft fix whatever the issue is.
I did suggest enabling file caching of the data archives. This might have some performance improvement on mechanical drive systems with a lot of memory. However that idea was shot down for the reasons stated above.
alt3rn1ty wrote:
Wed, 8. May 19, 12:03
If you compare this game with something like Skyrim Special Edition
It is stupid trying to even compare. Skyrim is an AAA budget game. If Egosoft had that many resources you could expect everything to be orders of magnitude bigger, better or faster. It was console optimized, specifically for 256MB Video Memory consoles like the Sony PlayStation 3, and probably had a team of people dedicated just to that aspect.

That said Skyrim had a ton of loading screens, especially when transitioning into dungeons. Most of the X4 loading issues could be solved by adding loading screens to mask them like Skyrim and other games did. Even games like Freelancer had loading screens when traveling between systems and the entire game installation was less than 1GB.
alt3rn1ty wrote:
Wed, 8. May 19, 12:03
but only needs up to 20% of disk IO access for that moment
What does this mean? Is this percentage time that the disk is scheduled for reading? Was this a mechanical drive or a SSD?

This could show that Skyrim was loading very inefficiently. For example if it used a queue depth of only 1 so could not keep the drive busy at all times. If it was on a SSD it could show that Skyrim has so little data to load that it could not keep the drive busy within the measurement window. Seeing how modern NVMe drives are capable of >1GB/sec random access in bursts this is very possible on such a drive seeing how small Skyrim is at any given time.
alt3rn1ty wrote:
Wed, 8. May 19, 12:03
.. Point being, never seen any issues loading textures in Skyrim Special Edition which has a vast amount of resources to juggle from HD into the VRAM space available on your graphics card. X4's needs in this regard are far less, and should be able to pre-cache a planet texture (even if you hit the Gate at speed on a highway) and load it instantly.

As I mentioned to CBJ earlier, I think the point at which the game starts pre-caching the next sector probably needs to be given a bit more of a run up, more distance to the gate if you are on a highway approaching the gate, as opposed to just flying into the gate normally, because when you are on a highway you hit the gate and close the distance faster than you do if not on a highway, and so the pre-caching of resources for the upcoming sector you jump to has less time to do the necessary pre-loading. Thats my theory anyway, and CBJ did neither confirm or deny .. Maybe its an embarrassing oversight for the programmers?
Problem is that just because you are near a gate does not mean you will enter it. Skyrim never had this problem as its huge overworld was continuous so it could stream in chunks in a lazy way long before the player approached the end of the world that was loaded. This was useful since it fitted the LOD model very well and there were no discontinuities needing a complete change in scenery. That said if you went stupidly fast or had IO contention then it was possible for loading screens to appear, and in its predecessor, Oblivion, you could literally walk off the edge of the map into the LOD background. Additionally Skyrim hid loading of objects by having them fade in, similar to games like Breath of the Wild and Xenoblade Chronicles (1, X and 2), which is not as obvious as the pop in that X4 does but ultimately gives the same playability.

And this is not even covering the obvious elephant in the room. When areas were discontinuous in Skyrim, eg entering a dungeon, then you were always confronted with a loading screen. If X4 had such loading screens there would be 0 pop in, just a lot of annoyed players complaining about loading screens.

Oh and let us not forget that Fallout 76 uses an engine based on that used by Skyrim. Fallout 76 is much more comparable because it is a similar generation of game so one can expect shaders and asset quality to have similar targets. We all know how bad that is with loading times. For example around launch entering the overworld from a dungeon took a good part of a minute, despite you having just entered that dungeon. Now one could forgive this if it was off a low RPM mechanical drive, but this was off a SATA 3.0 SSD. In comparison the loading performance of X4 is orders of magnitude better on the same medium.

Derp
Posts: 72
Joined: Thu, 9. Jul 15, 02:42

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Derp » Wed, 8. May 19, 21:30

Replace the white blotch with a planet-colored blotch or more space background, and have the planet fade in. If it takes a couple of seconds I doubt anyone will much care. It's the giant white ring large enough for a ghost to crawl out of that makes you stop and say "what the heck is that?" and really notice the planet appearing there that bugs people.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by SirNukes » Thu, 9. May 19, 02:01

alt3rn1ty wrote:
Wed, 8. May 19, 12:03
If you compare this game with something like Skyrim Special Edition, which has layers of textures applied to every object and even various parts (Head, Body, Hands, all different clothing and armour) of NPCs in game (Diffuse maps, Tangent Space Normal maps, Model Space Normal Maps, Specular maps, Glow maps, Reflectivity Maps, Environment Maps and Tone Maps) .. All of which have to be loaded from drive and decompressed from within an archive (BSA, Bethesda Softworks Archive).
Thankfully, Bethesda lets the OS cache their files. Which is really nice for tweaking mods, since restarting the game off an HDD takes only a 25% as long as the first startup (128 down to 33 seconds on my system).

Kernel Panic
Posts: 86
Joined: Tue, 2. May 06, 15:18
x3tc

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Kernel Panic » Thu, 9. May 19, 06:55

SirNukes wrote:
Thu, 9. May 19, 02:01
Thankfully, Bethesda lets the OS cache their files. Which is really nice for tweaking mods, since restarting the game off an HDD takes only a 25% as long as the first startup (128 down to 33 seconds on my system).
Linux has a filesystem cache by default (Windows does too, but I don't know a lot about it). Any reads/writes to/from a disk are saved in RAM until something else needs the memory. Which is why I am 95% positive that textures and sprites failing to load as soon as a sector is entered could be considered a bug. It seems to me like it's just a matter of raising the priority of what gets rendered first. I still think that the planet textures not loading isn't as important as being able to see my radar ASAP. I do like the idea of a "fade in" for planets so the giant white place holder is invisible to the user.

As far as mechanical HDD RAID vs a single SSD goes:

I don't think it matters much for a few reasons. Once things get loaded into memory they will stay there as long as the system has free RAM. So if you have enough RAM you only need to read from the disk(s) once. Any halfway decent OS will read (and cache) not just the the target sector it is seeking but also the sectors before and after it. This is because decades of statistics have shown that the kernel will almost certainly need to read one of those sectors very soon. Since my RAID was created in 512KB chunks it could take as few as 8000 sector reads to cache the entire X4 installation. In reality it's probably a few times that because not all of that will stay in my 16GB of RAM since the game does need quite a bit of memory to run the simulation. My point is that I am far from thrashing my disks on sector load. When I first installed the game I had it on a 256GB Samsung Evo SSD and it didn't seem to load sectors any better than my RAID. Launching the game or loading a save is much faster on my RAID than the SSD. On the other hand saving a game was much faster on my Evo 850. Go figure.

This might not be an issue for me in a month or two when I upgrade my system but I am not going to hold my breath because as I said before I think this caused by texture loading priorities set by the game engine. I don't think having 32GB of DDR4 will make it go away. It would be a nice surprise if it does though.

Imperial Good
Moderator (English)
Moderator (English)
Posts: 4750
Joined: Fri, 21. Dec 18, 18:23
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Imperial Good » Thu, 9. May 19, 14:29

Kernel Panic wrote:
Thu, 9. May 19, 06:55
Linux has a filesystem cache by default (Windows does too, but I don't know a lot about it). Any reads/writes to/from a disk are saved in RAM until something else needs the memory. Which is why I am 95% positive that textures and sprites failing to load as soon as a sector is entered could be considered a bug. It seems to me like it's just a matter of raising the priority of what gets rendered first. I still think that the planet textures not loading isn't as important as being able to see my radar ASAP. I do like the idea of a "fade in" for planets so the giant white place holder is invisible to the user.
X4 explicitly bypasses the file cache when reading game data files. This has been both confirmed by developers and by experimental testing. In fact it will even evict game data file pages from the file cache when the game starts.

Now this is the case for Windows. I am not sure if it also applies to the Linux build since that will be using different underlying IO calls with different behaviours.

For anyone wanting to confirm this behaviour on Windows, use RAMMap from Microsoft System Internals (free application) to inspect what data is loaded in the file cache. After X4 has been running a while one will notice that not a single page of any of the large data files is resident in the file cache. One can try to explicitly force all of the data files into the file cache using a custom written application and use RAMMap to verify their presence in the file cache. However after starting X4 and checking again all the previously cached data files will have been evicted from the file cache if X4 tried to access them.

This is the topic where it was discussed...
viewtopic.php?f=146&t=415226&p=4854045#p4854045

This is the quote...
CBJ wrote:
Tue, 12. Mar 19, 11:02
We have our own caching mechanisms which are designed to work with our catalog system, and make the OS caching redundant in most cases.

The 75s startup-time mentioned by the OP is not related to file caching, and would not be helped by enabling it. It's a separate issue that only seems to affect certain people, and is something we'll be looking into when we can.
The 75s startup-time was resolved by upgrading to an AHCI SSD, even though it is operating at just SATA 2.0 speed. Start up time is now sub 30 seconds.
Kernel Panic wrote:
Thu, 9. May 19, 06:55
It seems to me like it's just a matter of raising the priority of what gets rendered first. I still think that the planet textures not loading isn't as important as being able to see my radar ASAP. I do like the idea of a "fade in" for planets so the giant white place holder is invisible to the user.
Since moving to a AHCI SSD practically all pop in issues have out right vanished. Upon entering the system practically everything is immediately loaded at full quality. There might still be some if one teleports literally into the busiest sectors in the universe but that is kind of expectable given the nature of the problem.
Kernel Panic wrote:
Thu, 9. May 19, 06:55
I don't think it matters much for a few reasons. Once things get loaded into memory they will stay there as long as the system has free RAM. So if you have enough RAM you only need to read from the disk(s) once. Any halfway decent OS will read (and cache) not just the the target sector it is seeking but also the sectors before and after it. This is because decades of statistics have shown that the kernel will almost certainly need to read one of those sectors very soon. Since my RAID was created in 512KB chunks it could take as few as 8000 sector reads to cache the entire X4 installation. In reality it's probably a few times that because not all of that will stay in my 16GB of RAM since the game does need quite a bit of memory to run the simulation. My point is that I am far from thrashing my disks on sector load. When I first installed the game I had it on a 256GB Samsung Evo SSD and it didn't seem to load sectors any better than my RAID. Launching the game or loading a save is much faster on my RAID than the SSD. On the other hand saving a game was much faster on my Evo 850. Go figure.
Except X4 explicitly does not allow the OS to file cache the data files while X4 is running. Hence this mechanic does not apply for X4. The developers have no intention of allowing it to apply, I already asked a while ago and they gave the above response.

Again this is my observation from X4 running on Windows. If you are using Linux and if they do allow the use of the file cache on Linux then yes the performance difference will be quite small, especially with such large block size since one will be getting near sequential IO speed most of the time.

However on Windows the file cache is not used and so every asset loaded from disk will generate a sector read. Unless they use their own internal data file caching, and not the recommended approach of ready to use caching, most of the 512KB sector reads will be wasted as they will contain not the requested data so have to be discarded. This would mean that each asset request would generate at least 1 sector read and due to the random access nature of random sector reads on mechanical drives each might be subject to seek latency.
Kernel Panic wrote:
Thu, 9. May 19, 06:55
I don't think having 32GB of DDR4 will make it go away.
On Windows 32GB of DDR4 will not make a difference. Faster SSDs (NVMe) and CPUs (good single thread performance) might help reduce the prevalence or how noticeable asset loading is, at least on Windows.

User avatar
alt3rn1ty
Posts: 2298
Joined: Thu, 26. Jan 06, 19:45
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by alt3rn1ty » Thu, 9. May 19, 17:46

Guys, interesting topic you are creating inside my bug report, and I hate to disturb you showing off your technical knowledge, but would you kindly take it elsewhere if you are not helping the bug report in any way ?

As I said before "All I want to know is will Egosoft fix whatever the issue is."
Laptop Dell G15 5510 : Win 11 x64
CPU - 10th Gen' Core I7 10870H 2.2-5.0ghz, GPU - NVidia Geforce RTX 3060, VRAM 6gb GDDR5,
RAM - 32gb (2x16gb, Dual Channel mode set in BIOS) DDR4 2933mhz Kingston Fury Impact,
SSD - Kioxia M.2 NVME 512gb (System), + Samsung M.2 NVME 970 Evo Plus 1tb (Games)

:boron: Long live Queen Polypheides and may her tentacles always be supple.
Seeker of Sohnen.

User avatar
alt3rn1ty
Posts: 2298
Joined: Thu, 26. Jan 06, 19:45
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by alt3rn1ty » Sat, 20. Jul 19, 20:42

CBJ wrote:
Fri, 8. Mar 19, 09:40
The game does try to pre-load them shortly before you enter the sector, but there can be a lot to load and sometimes it can't get everything loaded in time. This is one of those things that should improve if you install the game on a faster disk such as an SSD. Even on an HDD you may be able to improve it somewhat by setting the game folder to be excluded from your AV software's checks, as this can sometimes slow down file access.
Update :

Now running X4 2.50 Hotfix 1 on This machine

8th Generation Intel® Core™ i7-8750H Processor (6-Core, 9MB Cache, up to 4.1GHz w/ Turbo Boost)
16gb DDR4 (2666MHZ) - Originally the machine came with 8gb which has been upgraded
NVIDIA® GeForce® GTX 1050 Ti - 4GB GDDR5 VRAM
128GB M.2 PCIe NVMe Solid State Drive - OS System drive, Clean install of Win 10 x64 1903
1TB WD Blue 3D NAND 2.5" SATA SSD - Originally the machine came with a HD which has been upgraded

Game installation drive is the WD Blue SSD
AV is windows Defender, set to ignore the X4 exe and the whole X4 Foundations folder aswell
The game is using the NVidia 1050 High Performance Graphics Adaptor

Just ran X4 Foundations 2.50 Hotfix 1, for about 4 hours.
CPU is occasionally peaking at 48% use, but mostly averages less than 40%, and does not need to use boost mode for this game so far.
NVidia GPU is running at a steady 35% use.
The machine is not even getting warm, and the fans have not had to kick into a higher speed.

TL:DR = The machine is running this game comfortably with plenty of grunt to spare in all departments.

Still experiencing the same issue, especially when coming out of a jump gate having enterred it on a highway.
Laptop Dell G15 5510 : Win 11 x64
CPU - 10th Gen' Core I7 10870H 2.2-5.0ghz, GPU - NVidia Geforce RTX 3060, VRAM 6gb GDDR5,
RAM - 32gb (2x16gb, Dual Channel mode set in BIOS) DDR4 2933mhz Kingston Fury Impact,
SSD - Kioxia M.2 NVME 512gb (System), + Samsung M.2 NVME 970 Evo Plus 1tb (Games)

:boron: Long live Queen Polypheides and may her tentacles always be supple.
Seeker of Sohnen.

User avatar
Pesanur
Posts: 1892
Joined: Sat, 5. Jan 08, 22:06
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Pesanur » Sun, 21. Jul 19, 21:58

They can add back the wormhole animation as a disguised loading screen while the planet textures are loaded.

reanor
Posts: 804
Joined: Thu, 23. Oct 03, 01:39
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by reanor » Tue, 30. Jul 19, 17:51

alt3rn1ty wrote:
Sat, 20. Jul 19, 20:42
CBJ wrote:
Fri, 8. Mar 19, 09:40
The game does try to pre-load them shortly before you enter the sector, but there can be a lot to load and sometimes it can't get everything loaded in time. This is one of those things that should improve if you install the game on a faster disk such as an SSD. Even on an HDD you may be able to improve it somewhat by setting the game folder to be excluded from your AV software's checks, as this can sometimes slow down file access.
Update :

Now running X4 2.50 Hotfix 1 on This machine

8th Generation Intel® Core™ i7-8750H Processor (6-Core, 9MB Cache, up to 4.1GHz w/ Turbo Boost)
16gb DDR4 (2666MHZ) - Originally the machine came with 8gb which has been upgraded
NVIDIA® GeForce® GTX 1050 Ti - 4GB GDDR5 VRAM
128GB M.2 PCIe NVMe Solid State Drive - OS System drive, Clean install of Win 10 x64 1903
1TB WD Blue 3D NAND 2.5" SATA SSD - Originally the machine came with a HD which has been upgraded

Game installation drive is the WD Blue SSD
AV is windows Defender, set to ignore the X4 exe and the whole X4 Foundations folder aswell
The game is using the NVidia 1050 High Performance Graphics Adaptor

Just ran X4 Foundations 2.50 Hotfix 1, for about 4 hours.
CPU is occasionally peaking at 48% use, but mostly averages less than 40%, and does not need to use boost mode for this game so far.
NVidia GPU is running at a steady 35% use.
The machine is not even getting warm, and the fans have not had to kick into a higher speed.

TL:DR = The machine is running this game comfortably with plenty of grunt to spare in all departments.

Still experiencing the same issue, especially when coming out of a jump gate having enterred it on a highway.
Don't use external drives... And as a word of advise, considering that X4 is one of the heaviest games on reading data from the drive, and if your motherboard supports it, I really recommend to get SSD M.2 drive. They are a bit more expensive than regular SSDs, but a lot faster for reading/writing. Also, according to the PC stats you posted and the below is a result of the bottleneck calculator. You need a better GPU for your setup, you have quite a high data bottleneck
Average bottleneck percentage: 94.91%
This result is based on average CPU and GPU usage from different programs and games. It changes based on operating system, background processes activity and targeted applications. This result is not universal and changes based on differences in hardware and software enviroments. Please do not use this calculator primary as decision maker than as helping tool to understand performance correlations between different components.

Your graphic card is too weak for this processor.
Intel Core i7-8750H (Clock speed at 100%) with NVIDIA GeForce GTX 1050 Ti (Clock speed at 100%) x1 will produce 94.91% of bottleneck. Everything over 10% is considered as bottleneck.

We recommend you to replace NVIDIA GeForce GTX 1050 Ti with NVIDIA GeForce GTX 1070
https://pc-builds.com/calculator/Core_i ... d0VZlu/16/

Follow their advise and get at least that GPU and you'll see a lot of performance increase. And of course the M.2 SSD wouldn't hurt either.
“The dark and the light, they exist side by side." ... “It is often in the darkest skies that we see the brightest stars."

User avatar
alt3rn1ty
Posts: 2298
Joined: Thu, 26. Jan 06, 19:45
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by alt3rn1ty » Tue, 30. Jul 19, 18:09

reanor wrote:
Tue, 30. Jul 19, 17:51
Don't use external drives... And as a word of advise, considering that X4 is one of the heaviest games on reading data from the drive, and if your motherboard supports it, I really recommend to get SSD M.2 drive.
You quoted my post but apparently did not read it.

1. I do not have any external drives (well apart from the one USB drive I plug in occasionally to backup documents)
2. I have an M.2 SSD as my system drive.
3. The game is installed on the INTERNAL 1tb WD Blue SSD
4. As per the link in my post, its a laptop, GPU is not easily upgraded, but handles this game well enough

None of anything you mention has any bearing on this issue, the devs have already admitted it can be a problem, and this machine in all respects is above minimum spec, even for a laptop GPU which is less capable than a desktop card of the same model with more bandwidth.

And as I said earlier, given the promulgated specifications on the site I bought the game from .. https://www.gog.com/game/x4_foundations

.. I believe I have every reasonable expectation this game should work fine on the machine I have.

As far as the GPU being a bottleneck, thats why I run the game with lower screen res than native, and tweak down custom settings (from what the game engine recommends for this machine) to give a consistantly good FPS. This game series is (and always has been) heavier on the CPU needs than it is on the GPU. I concur that the GPU for X4 is more important these days. Edit : PS, the core clock frequency of the 1050 GPU you compaired with is a lot lower than the GPU in my gaming machine, which has excellent cooling (the laptop does not even get warm if I ramp up the game settings, and the fans are running at a quarter of their full speed in all my testing so far). GPU usage on this machine has never gone above 45% with X4.

Hence to me this issue is a bug with the game.
Last edited by alt3rn1ty on Tue, 30. Jul 19, 19:21, edited 4 times in total.
Laptop Dell G15 5510 : Win 11 x64
CPU - 10th Gen' Core I7 10870H 2.2-5.0ghz, GPU - NVidia Geforce RTX 3060, VRAM 6gb GDDR5,
RAM - 32gb (2x16gb, Dual Channel mode set in BIOS) DDR4 2933mhz Kingston Fury Impact,
SSD - Kioxia M.2 NVME 512gb (System), + Samsung M.2 NVME 970 Evo Plus 1tb (Games)

:boron: Long live Queen Polypheides and may her tentacles always be supple.
Seeker of Sohnen.

Imperial Good
Moderator (English)
Moderator (English)
Posts: 4750
Joined: Fri, 21. Dec 18, 18:23
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by Imperial Good » Tue, 30. Jul 19, 18:48

reanor wrote:
Tue, 30. Jul 19, 17:51
I really recommend to get SSD M.2 drive. They are a bit more expensive than regular SSDs, but a lot faster for reading/writing.
This makes little difference over a SATA SSD. Most of the improvement comes from the random access latency and bandwidth, which even a SATA SSD does orders of magnitude better than a mechanical drive. The improvement from SATA SSD (AHCI) to PCIe SSD (NVMe) is nowhere near as great. Few applications have significant gains from PCIe SSDs and those that do are usually productivity related. It is unlikely a person could tell the difference between them from black box testing the game experience.

User avatar
alt3rn1ty
Posts: 2298
Joined: Thu, 26. Jan 06, 19:45
x4

Re: [BUG] Any version - Planets slow to render when entering sector

Post by alt3rn1ty » Wed, 31. Jul 19, 18:36

Just for any more doubting Thomas's I made a video with MSI Afterburner running.

Just for the hell of it, I also ramped up the game settings to what it recommends for my machine (without any of my down tweaks to help the laptops life expectancy), lets give this GPU a bit of a workout ..

To see the glitch happening with planet loading, go to timeline 2 minutes and 15 seconds. MSI Afterburner lags a bit because it only polls every 1000 milliseconds, but at that timeline where the glitch occurs just after jumping through the next gate, you will notice that GPU use does not go above 60%

https://youtu.be/kLP_UE684W4

@reanor you will also notice the GPU core clock speed 1734mhz - A little bit faster than the 1290mhz 1050 you compaired with on the Bottleneck Calculator. And no, I have not OC'd the GPU.

As they say on the site "*This result is based on average CPU and GPU usage from different programs and games. It changes based on operating system, background processes activity and targeted applications. This result is not universal and changes based on differences in hardware and software enviroments. Please do not use this calculator primary as decision maker than as helping tool to understand performance correlations between different components."
Laptop Dell G15 5510 : Win 11 x64
CPU - 10th Gen' Core I7 10870H 2.2-5.0ghz, GPU - NVidia Geforce RTX 3060, VRAM 6gb GDDR5,
RAM - 32gb (2x16gb, Dual Channel mode set in BIOS) DDR4 2933mhz Kingston Fury Impact,
SSD - Kioxia M.2 NVME 512gb (System), + Samsung M.2 NVME 970 Evo Plus 1tb (Games)

:boron: Long live Queen Polypheides and may her tentacles always be supple.
Seeker of Sohnen.

Post Reply

Return to “X4: Foundations - Technical Support”