Please help: Object3D.clone() allocates Mbs of RAM each time

[Dominic]

Hello,
       I have a big memory problem with JPCT. I need to create multiple Object3D from the same model (Mesh). The model comes from a .3ds file. When I do an Object3D.clone() or a Object3D.Object3D(Object3D), the mesh information is referenced in the clone, instead of copied, as it should be. However, there is a Vectors object in Object3D. This object is copied in the clone instead of referenced, and it uses LOTS of memory since it contains 25 big arrays, a few Mbs for each clone. :!: When I create multiple instances this way, it easily adds up to hundreds of Mbs for just 60 Objects and over a 1 Gb for 500 Objects :shock: ! Is there a way to fix this ? Our project can't work this way, it just uses way too much memory. Can't those arrays be shared between Object3D instances or even deactivated. (What are they used for anyway ? Isn't the Mesh object enough to describe the model ? )

I'm using the OpenGL renderer.
Config.saveMemory = true;    // Doesn't help.
Config.glVertexArrays = false;    // Doesn't help.
Config.polygonBufferSize = 1;    // Doesn't help either.

Please help. Thank you very much.

[EgonOlsen]

No, they can't be shared. Mesh may be almost enough to describe the model, but not to render it. Config.saveMemory already decreases the amount of memory an Object3D needs...but that's obviously not enough in your case.
May i ask how big this model is?

[Dominic]

No, they can't be shared. Mesh may be almost enough to describe the model, but not to render it. Config.saveMemory already decreases the amount of memory an Object3D needs...but that's obviously not enough in your case.
May i ask how big this model is?

The model is a bit big, the .3ds file contains 31 objects. I do have a similar problem with less complex models, to a lesser extent.

But still, this doesn't seem normal to me to have all this memory allocated for each Object. All objects are identical, I see no reason why vertex information shouldn't be shared. Are those buffers for transformed vertices ? In that case, can't a single buffer be used for every objects ?

I hope there is a solution to this.

[EgonOlsen]

Vertex information is shared, because that's what's inside Mesh. Vectors contains transformed vertices and normals, texture coordinates, color information etc...everything that can be different from object to object (it may not always be different, but what should one do if it may...).

I'm not sure if this is really solvable. It may be possible to get rid of some of the information in same situations, but i've already reduced it to what i considered to be necessary. Maybe you can upload the model in question (or a similar complex one), so that i can see for myself?

[Dominic]

Vertex information is shared, because that's what's inside Mesh. Vectors contains transformed vertices and normals, texture coordinates, color information etc...everything that can be different from object to object (it may not always be different, but what should one do if it may...).

I'm not sure if this is really solvable. It may be possible to get rid of some of the information in same situations, but i've already reduced it to what i considered to be necessary. Maybe you can upload the model in question (or a similar complex one), so that i can see for myself?

I don't understand, once you have transformed the Mesh and displayed it, why couldn't the buffers containing the T&L vectors be reused by other objects ?

Could it be possible to display the same Object3D at multiple positions at once ?
Merging the objects in the .3ds may reduce memory usage. The total of the buffers in the Vectors object is only 37% of the total of the buffers in the 31 objects. But I didn't have time to really try that yet, I would need to redo a bit of code. Not to mention that it would not really fix the problem.

[EgonOlsen]

I don't understand, once you have transformed the Mesh and displayed it, why couldn't the buffers containing the T&L vectors be reused by other objects ?

No, because that's not the way jPCT works. The result of the T&L stuff is a list (the VisList) with indices into the Object3Ds' structure to indicate which polygons of which objects should be drawn. Therefor, each object has to keep this information. I'm already reusing some information on each object which was formerly located in Vectors too but is now in World.
Are you using the OpenGL renderer only? If so, i can imagine that i could get rid of some information at runtime "the hard way", i.e. nulling the instances on demand. This will of course cause a crash when using software renderer, but may work for hardware...

[cyberkilla]

Roughly how many polygons are there in the file?

[Dominic]

I don't understand, once you have transformed the Mesh and displayed it, why couldn't the buffers containing the T&L vectors be reused by other objects ?

No, because that's not the way jPCT works. The result of the T&L stuff is a list (the VisList) with indices into the Object3Ds' structure to indicate which polygons of which objects should be drawn. Therefor, each object has to keep this information. I'm already reusing some information on each object which was formerly located in Vectors too but is now in World.
Are you using the OpenGL renderer only? If so, i can imagine that i could get rid of some information at runtime "the hard way", i.e. nulling the instances on demand. This will of course cause a crash when using software renderer, but may work for hardware...

For the moment I'm using the the OpenGL renderer only, since I have issues with the software renderer (textures don't tile and lighting is much brighter), but that's another story and less of a priority for now. Would nulling the instances of those arrays require a new version of JPCT ?

I don't know how many polygons there are in the file. They are not my creation. But there are a bit too much. But the same models were used without problems with another engine. However, models with a much more "normal" number of polygons have the same issue in JPCT, to a much lower extent. By reading the forum, I could see that I'm not the first to think that JPCT uses a lot of RAM. Storing each individual visible polygon in the VisList may be the source of this high memory requirement.

[cyberkilla]

I can understand your concern.

I dont see any problem as yet, but I havent really paid attention to the ram usage, if I am honest with myself;)

[Dominic]

I guess you will be interested to know that merging the objects in my 3ds file partially solved the problem. It requires less memory now.

The bad news is that it still requires a lot of memory and it could still be a problem.
If I understand correctly what you said, JPCT calculate the T&L stuff of all objects and after it has done so for every object it starts a rendering loop, rendering each polygon. Did I understand correctly ? If it is the case, doing so probably is an obstacle to performance as well as memory usage. If that is how JPCT work, then the T&L information of an object is pretty much guaranteed to no longer be in the CPU's cache when rendering of this object is done. It's place in the cache has been taken by the T&L of other objects­. This cause more transfers from the main RAM (wish is much slower than most people think) to the CPU cache than it would do if rendering of an object was done immediately after calculating the T&L for this object. Of course, you would need to sort your objects by material then, to ensure performance in hardware rendering mode. Since your textures a stored per polygon, this could be a problem.

[EgonOlsen]

The VisList isn't storing the polygonal information, it just references it. The VisList itself really isn't a memory hog. The "problem" is the hybrid approach jPCT uses, i.e. combining a hardware and software renderer by using a basically software renderer based pipeline with some optimizations for hardware. This isn't a solely hardware based polygon pusher, which has its pros but also its cons and one of them may be a higher memory usage (albeit i've never noticed jPCT taking more memory than other Java based engines, but that's another story and doesn't help you).
I *think* that it could be possible to move some stuff out of Vectors and share it between objects, because it's not needed at render time (as far as i can see...). I'll try this to see if it's possible and if it helps.
About caching and the software renderer: I wouldn't worry about the cpu's cache. That really isn't a problem. And the software renderer not supporting tiling and being to bright is caused by the legacy renderer being used (which is default until now but will change in the next version). Just enable the software renderer in OpenGL mode and you should be fine.

[EgonOlsen]

I've uploaded a beta version with reduced memory footprint of the Vectors class. I think that this is already close on what can be done in this regard for this class. Give it a try: http://www.jpct.net/download/beta/jpctapi2.zip

[Melssj5]

Is there a way to render the same object3D several times on different places to save memory. Defininf it once and adding it to several places like blocks to reuse. Thjat would help a lot for saving memory and working for some kind of games.

[cyberkilla]

You could have a transformation array, to move it around