www.jpct.net
jPCT - a 3d engine for Java => Support => Topic started by: Mr_Chaos on February 07, 2015, 07:51:06 pm
-
So I have a texture that is sometimes replaced, but it seems to leak Native memory (Non heap).
I do the following
1) unload texture
2) New texture
3) Replace texture
4) Buffer.display
Is this wrong ?
-
Should be fine unless you render some object that uses the unloaded texture in between. This can happen when using a multi-threaded renderer like the AWTGLRenderer or the multithreading option of the normal renderer. In that case, try to synchronize the whole texture replacement code on the lock object that you can obtain from the FrameBuffer.
What exactly makes you think that you are losing native memory?
BTW: You can set the Logger to debug and Config.glVerbose to true to keep track of texture up- and unloading, if that helps.
-
We are using the AWTGLRenderer, so that might explain it, but I am doing this in the rendering loop, so was hoping everything was synchronized allready.
I can see that I am leaking handles and my virtual memory skyrockets, after 1 hour of running I am using 6GB after 1,5 12GB and then the systems stops completely.
I'll try the FrameBuffer.synchronization and the Config.glVerbose.
-
I get this
Allocating 4194304 bytes of direct memory for texture: com.threed.jpct.Texture@25bbf0b7
Creating new disposable buffer of size 4194304
Caching 4194304 bytes of direct memory!
But nothing about unloading a texture
this is the code
_textureManager.unloadTexture(_buffer, _elementsTexture);
_elementsTexture = new Texture(_elementsImage);
_elementsTexture.setMipmap(false);
_elementsTexture.setGLFiltering(false);
_textureManager.replaceTexture(ELEMENTS_TEXTURENAME, _elementsTexture);
-
But that's not the complete log or is it? It doesn't contain anything about uploading the new texture either. Both should happen during the next render pass.
-
But that's not the complete log or is it? It doesn't contain anything about uploading the new texture either. Both should happen during the next render pass.
This is all I get when changing the Texture
Allocating 4194304 bytes of direct memory for texture: com.threed.jpct.Texture@27174693
Creating new disposable buffer of size 4194304
Caching 4194304 bytes of direct memory!
updateElementsTexture <- My own log
Loading Texture...from Image
-
Forgot
Logger.setLogLevel(Logger.LL_VERBOSE);
But still this is what I get
updateElementsTexture
Loading Texture...from Image
Allocating 4194304 bytes of direct memory for texture: com.threed.jpct.Texture@32cb56e6
Caching 4194304 bytes of direct memory!
-
I'll create my own test case and see what happens. I'm not sure how upload plays together with replace...i'll report back.
-
BTW: Log level should be debug, not just verbose.
-
BTW: Log level should be debug, not just verbose.
Tried with both, wasn't sure which was most important
-
I think i can reproduce the issue...i'm just not sure what causes it. Looks like as if the unload operation never happens. The log output is sparse as you've already mentioned. I thought that it's on par with jPCT-AE, which is much more verbose in debug mode, but obviously it isn't. I'll add more logging while i'm on it and report back later...
BTW: It works as expected with the native GL renderer, so it has something to do with AWT renderer's multithreaded nature... ???
-
Well, actually it's all fine...my test case was just ignoring the fact that the draw loop executes in parallel to the rendering thread, so you can in theory stuff thousands of new and never used textures into the unload-queue before the first render pass is able to dispose them all. Once i handled this, it behaved as expected. However, i'm not sure if there isn't some similar side-effect in your case, so i suggest to try what i did in this test case: Move the texture replacement stuff into the finishedPainting()-method of an IPaintListener implementation. Have a look here:
import java.awt.Canvas;
import java.awt.Color;
import java.util.Random;
import javax.swing.JFrame;
import com.threed.jpct.Config;
import com.threed.jpct.FrameBuffer;
import com.threed.jpct.IPaintListener;
import com.threed.jpct.IRenderer;
import com.threed.jpct.Logger;
import com.threed.jpct.Object3D;
import com.threed.jpct.Primitives;
import com.threed.jpct.Texture;
import com.threed.jpct.TextureManager;
import com.threed.jpct.World;
public class TextureModTest implements IPaintListener {
private static final long serialVersionUID = 1L;
private Random rnd = new Random();
private int cnt = 0;
private World world;
private FrameBuffer buffer;
private Object3D box;
private Texture boxTex;
private JFrame frame;
public static void main(String[] args) throws Exception {
new TextureModTest().loop();
}
public TextureModTest() throws Exception {
Logger.setOnError(Logger.ON_ERROR_THROW_EXCEPTION);
Logger.setLogLevel(Logger.LL_DEBUG);
Config.glVerbose = true;
frame = new JFrame("Hello world");
frame.setSize(800, 600);
frame.setVisible(true);
frame.setDefaultCloseOperation(JFrame.HIDE_ON_CLOSE);
world = new World();
world.setAmbientLight(0, 255, 0);
boxTex = new Texture("box.jpg");
TextureManager.getInstance().addTexture("box", boxTex);
box = Primitives.getBox(13f, 2f);
box.setTexture("box");
box.setEnvmapped(Object3D.ENVMAP_ENABLED);
box.build();
box.compile();
world.addObject(box);
world.getCamera().setPosition(50, -50, -5);
world.getCamera().lookAt(box.getTransformedCenter());
}
private void loop() throws Exception {
buffer = new FrameBuffer(800, 600, FrameBuffer.SAMPLINGMODE_GL_AA_4X);
Canvas canvas = buffer.enableGLCanvasRenderer();
buffer.disableRenderer(IRenderer.RENDERER_SOFTWARE);
frame.add(canvas);
buffer.setPaintListener(this);
while (frame.isShowing()) {
box.rotateY(0.01f);
buffer.clear(java.awt.Color.BLUE);
world.renderScene(buffer);
world.draw(buffer);
buffer.update();
buffer.displayGLOnly();
canvas.repaint();
}
System.exit(0);
}
@Override
public void startPainting() {
// TODO Auto-generated method stub
}
@Override
public void finishedPainting() {
TextureManager.getInstance().unloadTexture(buffer, boxTex);
boxTex = new Texture(256, 256, new Color(rnd.nextInt(256), rnd.nextInt(256), rnd.nextInt(256)));
TextureManager.getInstance().replaceTexture("box", boxTex);
System.out.println("Iteration: " + cnt++ + "/" + Runtime.getRuntime().maxMemory() + "/" + Runtime.getRuntime().freeMemory());
}
}
Apart from that, i've uploaded a new beta jar with better debug logging. If the IPaintListener stuff doesn't help, maybe this will help to track down the problem: http://jpct.de/download/beta/jpct.jar (http://jpct.de/download/beta/jpct.jar)
-
Ok, the logs helped a bit, but I cannot figure out what is wrong.
It seems that whenever i replace the texture the old texture is uploaded as well as the new one.
So the leak is that both the old and the new texture exists ... or something.
Using the taskmanager I can see the handle count increate whenever i replace the texture
Texture 2 added for unloading (1)!
Loading Texture...from Image
Textures to unload: 1
Unloading texture 2
texture 2 unloaded from GPU memory!
Allocating 4194304 bytes of direct memory for texture: com.threed.jpct.Texture@76ce21e7
Creating new disposable buffer of size 4194304
Caching 4194304 bytes of direct memory!
New texture's id is: 2
New texture uploaded: com.threed.jpct.Texture@76ce21e7 in thread Thread[AWT-EventQueue-0,6,main]
New texture's id is: 6
New texture uploaded: com.threed.jpct.Texture@51a3adb in thread Thread[AWT-EventQueue-0,6,main]
updateElementsTexture
Texture 6 added for unloading (1)!
Loading Texture...from Image
Textures to unload: 1
Unloading texture 6
texture 6 unloaded from GPU memory!
New texture's id is: 6
New texture uploaded: com.threed.jpct.Texture@51a3adb in thread Thread[AWT-EventQueue-0,6,main]
New texture's id is: 7
New texture uploaded: com.threed.jpct.Texture@66c9cfd2 in thread Thread[AWT-EventQueue-0,6,main]
-
I'm not sure what i'm seeing there. It somehow looks as if the unloaded texture will be uploaded again right afterwards, but i'm not entirely sure. Can you add some logging on your side so that we can keep track of which texture should actually be unloaded and which one is the replacement texture? It should be sufficient to output the Texture instance itself, so that we can simply compare the memory addresses of those with the ones logged by the renderer.
-
Ok, by changing it to use the IPaintListener the logs look a lot better, the problem is that I still have a handle leak.
I am using a BufferedImage to create the texture from, don't know if that might be a problem ?
********************updateElementsTexture********************
oldTexture=<com.threed.jpct.Texture@24fa5c93>
Texture 2 added for unloading (1)!
Loading Texture...from Image
newTexture=<com.threed.jpct.Texture@324895c0>
********************updateElementsTexture********************
Textures to unload: 1
Unloading texture 2
texture 2 unloaded from GPU memory!
New texture's id is: 2
New texture uploaded: com.threed.jpct.Texture@324895c0 in thread Thread[AWT-EventQueue-0,6,main]
-
I don't think that it's related to BufferedImages. Are these handles lost forever or will they be reclaimed when garbage collection kicks in? Apart from that, i've identified the problem that causes the re-upload of actually unloaded textures. It's related to the double buffering of rendering commands that the AWTGLRenderer does. I'll try to find a fix for it this evening. Maybe that fixes the handle problem as well.
-
They are lost forever, never reclaimed.
When my program reached about 2500 handles it's using > 5GB native memory
-
What puzzles me is that it actually uses native memory. If it's really a texture-remains-on-the-gpu-problem, i would expect it to eat up the gpu memory and not the main memory and you won't see any OS memory handles for that (maybe unless it's some onboard chipset...). And if it just keeps the reference to the Texture instance, it would consume VM memory, not native memory. The only native memory that's being used when dealing with textures is the memory that's used to upload them...and that's only a temp buffer, it's not persistent. And even if it were, i would expect it to be counted as VM memory as well because it's reserved by it.
Anyway, i've uploaded a new jar: http://jpct.de/download/beta/jpct.jar (http://jpct.de/download/beta/jpct.jar). This one fixes two potential problems:
- Textures get unloaded that have never been unloaded. In former versions, such textures would have remained in the unload queue forever, because you can't be sure that it won't be uploaded in the next frame. In this version, these references will be removed after a short timeout.
- Textures that are still present in one of the AWTGLRenderer's buffers. In former versions, these would have been unloaded->uploaded->forgotten on the gpu. Now, the unloading will be delayed until they are no further references to them in the buffers.
Both operations will be logged in debug mode if they happen.
I'm not sure if this helps in your case, but it's worth a try. If it doesn't help: Have you tried not to use BufferedImage? Just to be sure...
-
It works now, handle count is completely stable and the same is my Memory usage
-
Cool! I'm glad we found it. I still don't get why it consumed native memory instead of GPU memory, but anyway...