[question]remove element from array...

[Orfevs]

Me and my arrays. What can I say, I love the stuff..

When using

Code: Select all

remove element from array $array at index $index

...and the element is another array;-
Will the sub array be freed up - or left as occupied space to bloat the saves?
Anyone?

[Shush]

Me and my arrays. What can I say, I love the stuff..

When using

Code: Select all

remove element from array $array at index $index

...and the element is another array;-
Will the sub array be freed up - or left as occupied space to bloat the saves?
Anyone?

It just deletes the reference in your array to that other array, so yes the other array still exists in memory.

Well that's how it used to work, I haven't checked if the last couple of years worth of patches changed this.

[zanzal]

I assume there is some form of garbage collection, because if there is not then lots of bad things must eventually happen. So if nothing persistent references your array then it must at some point it must get freed. One has to wonder how it would work that the data should be saved when nothing references it. Saving an entire virtual machine state garbage and all seems unlikely. Far more likely it is that anything not referenced eventually gets cleared out even if that only happens during a save/load cycle. If not then scripts allocating arrays should eventually destroy all save games.

My guess is that your orphaned array might stick around for a little while, but not beyond a save and load.

[Orfevs]

AFAIK there's a garbage collection. I believe using "Reinit script caches" cleans out the trash, but then again, I'm not using MSCI so I don't know much about it.

To be on the safe side, I just unravel the arrays recursively before removing the references.

[Shush]

Far more likely it is that anything not referenced eventually gets cleared out even if that only happens during a save/load cycle. If not then scripts allocating arrays should eventually destroy all save games.

It's all written in C/C++ and I am going to guess here without smart pointers or any form of RAII and obviously without a GC. But the KC that sits on top of the C/C++ and underneath the MSCI and MD might have its own rudimentary GC.

My guess is that your orphaned array might stick around for a little while, but not beyond a save and load.

This is how I would suggest it's handled, it gives you a small risk of running out of memory during any one session, but automatically releases all variables/arrays in memory that aren't defined as globals on the next save/load.

AFAIK there's a garbage collection. I believe using "Reinit script caches" cleans out the trash, but then again, I'm not using MSCI so I don't know much about it.

With respect to MSCI scripts, reinit script caches is the same as loading a saved game/starting a new game in that all the scripts get re-loaded. So if Zanzal's and my conjecture is correct it will have the same effect, i.e. releasing of non global variables

To be on the safe side, I just unravel the arrays recursively before removing the references.

Yup this is exactly what I do.

[Jack08]

Your orphaned array is cleared the next time the GC is run, so long as there are no more references to it. It may even free automatically with reference counting.

An easy way to see exactly whats happening would be to make a loop that creates hundreds of arrays, and then orphans them, and watch the memory usage of the process.

Either way, no matter what happens, no - your not going to create a memory leak, The only way you could ever create a memory leak would be to have arrays that have references to themselves within themselves, or have arrays that reference each other. In the case of an self-referencing array, or an set of arrays that reference each other, so long as all refreshes to these arrays from the outside are freed, they wont be saved with the savegame and the memory freed when the game reboots.

Reinit script caches simply reloads the command tables and re-executes setup scripts, it has nothing to do with the GC, its just a function within the script editor, infact the function itself should probably be named something else as its current name is misleading.

The game holds your hand majorly, as far as i know, it is impossible to create a permanent memory leak in your save game, unless you go infinitely creating and storing arrays without ever derefrencing them, like ... i dont know, randomly generating global variable names and creating arrays on each them - add infinity, with run away sub tasks constantly populating them... but why would you do that? bahahaha...

I use an absurd amount of dynamically created and destroyed arrays in my coding, because i cant make classes and structures - and ive never once ran into memory issues, and ive never unraveled an array - you can have arrays 10 levels+ deep and they will still correctly free if you orphan the parent array.

[Orfevs]

Your orphaned array is cleared the next time the GC is run, so long as there are no more references to it. It may even free automatically with reference counting.

An easy way to see exactly whats happening would be to make a loop that creates hundreds of arrays, and then orphans them, and watch the memory usage of the process.

Call me lazy, but ..aw hell. Just call me lazy.

Either way, no matter what happens, no - your not going to create a memory leak, The only way you could ever create a memory leak would be to have arrays that have references to themselves within themselves, or have arrays that reference each other. In the case of an self-referencing array, or an set of arrays that reference each other, so long as all refreshes to these arrays from the outside are freed, they wont be saved with the savegame and the memory freed when the game reboots.

I make a rule out of not making self referencing arrays to avoid such problems in the first place.

Reinit script caches simply reloads the command tables and re-executes setup scripts, it has nothing to do with the GC, its just a function within the script editor, infact the function itself should probably be named something else as its current name is misleading.

Yep, that's pretty misleading. So the garbage is only cleared on load then?

I use an absurd amount of dynamically created and destroyed arrays in my coding, because i cant make classes and structures - and ive never once ran into memory issues, and ive never unraveled an array - you can have arrays 10 levels+ deep and they will still correctly free if you orphan the parent array.

You're saying as long as the arrays aren't global, they'll lose scope and die once the script returns? Because I always unravel temporary arrays I allocate specifically (i.e not menu arrays). If the space is freed only after reloading, then I'll continue to unravel them since that will only build up - especially with multiple plugins installed that may or may not release them in the same manner.

[Jack08]

You're saying as long as the arrays aren't global, they'll lose scope and die once the script returns? Because I always unravel temporary arrays I allocate specifically (i.e not menu arrays). If the space is freed only after reloading, then I'll continue to unravel them since that will only build up - especially with multiple plugins installed that may or may not release them in the same manner.

Sort of. Local variables in scripts are - ironically - just values within an array, within the script array. When the script ends, that script is de-referenced, which frees the reference to all variables within it, which frees all the variables when the GC runs - if your script has passed that array to another script, who is still running, then there is still an active reference to your array. When said script ends, then your array is freed.

If your array is added to a global variable, and never removed from it - it never goes out of scope, and is never freed.

The GC runs in either in real-time or when it needs to, i dont know for sure, but not just at load or at sector change - if it didn't my own mods wouldn't be stable for 30+ days on end without changing sector or reloading, or saving.

By unraveling arrays, i assume you mean setting everything to 0 and then resizing the array to 0 and then zeroing the actual $Array? This doesn't free any memory at all. In fact there's nothing you can do in the script engine to absolutely force the freeing of memory, the game will do it when the game wants to do it, all thats happened here is forced the game to mark all the relevant memory for that array for deletion when the GC next runs.

Take this simple loop for example:

Code: Select all

while [TRUE]
   $Array = array alloc: size=100
   = wait 100 ms
end

Each time $Array is overwritten, the value previously in $Array gets marked for deletion as its reference counter has hit 0, i dont believe it happens then and there, but it also wont make the game build up memory until it crashes.

My point is, even thinking about memory management in MSCI is wasted effort, the only way your going to make a script use a worrying amount of memory is if you do something that keeps lots and lots of variables in scope forever, so long as things go out of scope - things get freed, in the example above, every loop, $Array should be considered going out of scope, then created again.

[Orfevs]

When the script ends, that script is de-referenced, which frees the reference to all variables within it, which frees all the variables when the GC runs

Sound logical enough. Cannot make a script language without trying to cover this area, otherwise things will go boom soon.

- if your script has passed that array to another script, who is still running, then there is still an active reference to your array. When said script ends, then your array is freed.

That's a chapter in itself. Passing local arrays back and forth isn't a good thing. I'm pretty certain what you send is a copy rather than a reference to the calling script's array. That kind of array handling has caused me a lot of grief. While I'm no longer a coder, I am used to both high and low level programming, just not as much on the PC.

If your array is added to a global variable, and never removed from it - it never goes out of scope, and is never freed.

At least not until I do it myself.. The biggest arrays of mine are created during setup and freed during uninstall. But I do wonder why my hotkeys remain even after unregistering.

The GC runs in either in real-time or when it needs to, i dont know for sure, but not just at load or at sector change - if it didn't my own mods wouldn't be stable for 30+ days on end without changing sector or reloading, or saving.

Well, that means I can shorten my scripts somewhat. Need the speed anyhow.

By unraveling arrays, i assume you mean setting everything to 0 and then resizing the array to 0 and then zeroing the actual $Array?

More or less. Resizing first.. then nulling the references recursively. As should be with good programming practice. Does not apply to game scripting, obviously.

This doesn't free any memory at all. In fact there's nothing you can do in the script engine to absolutely force the freeing of memory, the game will do it when the game wants to do it, all thats happened here is forced the game to mark all the relevant memory for that array for deletion when the GC next runs.

So resizing an array to 0 just places the max pointer to offset 0?
Nevermind.. doesn't matter anyhow. I never create dynamic arrays big enough to topple the game, and the global arrays are static. Mostly.
I estimate the lot to about 75KB.

Take this simple loop for example:

Code: Select all

while [TRUE]
   $Array = array alloc: size=100
   = wait 100 ms
end

Each time $Array is overwritten, the value previously in $Array gets marked for deletion as its reference counter has hit 0, i dont believe it happens then and there, but it also wont make the game build up memory until it crashes.

I always figured it happens when the script loses its scope. It's food for thought, I'll give you that. Setting that thing loose with infinite loop detection will be an interesting experiment.

My point is, even thinking about memory management in MSCI is wasted effort, the only way your going to make a script use a worrying amount of memory is if you do something that keeps lots and lots of variables in scope forever, so long as things go out of scope - things get freed, in the example above, every loop, $Array should be considered going out of scope, then created again.

Thinking about it sets my head spinning for sure. In any case, I always try to keep variables to a minimum and reusing as much as possible.
Thanks for clearing things up. Those damnable arrays have been bugging me since I started scripting X3TC! In one way or another.

[Jack08]

That's a chapter in itself. Passing local arrays back and forth isn't a good thing. I'm pretty certain what you send is a copy rather than a reference to the calling script's array. That kind of array handling has caused me a lot of grief. While I'm no longer a coder, I am used to both high and low level programming, just not as much on the PC.

When an array is created, all you receive is the pointer, you never actually receive the actual object, meaning no matter what you do with it - pass it to other scripts, set it as a variable, etc - its always pointing to the same section of memory.

This is why you can modify an array within a global/local/argument variable, without having to write that array back to the global/local/argument variable after modifying it. When passing arrays around your simply copying the pointer to it, rather then it entirely.

Now this only holds true for Arrays, as arrays are passed by pointer, everything else (ints, strings) is passed by value and copied.

While "Pointer" isnt a truly accurate example of whats happening - its more along the lines of an ID number, if the game was to save a pointer into the save game that would be disasterous - it gets the point across (Excuse the pun!)

[Orfevs]

No wonder script languages are confusing.. I just need to tell myself repeatedly that it isn't C/C++ or assembler. Old habits die hard though.