Author Topic: Out of memory  (Read 12856 times)

Offline Kaiidyn

  • long
  • ***
  • Posts: 103
    • View Profile
Out of memory
« on: February 24, 2011, 02:41:09 pm »
I'm starting to get really frustrated with this error...

Code: [Select]
Out of memory on a 3527724-byte allocation.
 FATAL EXCEPTION: GLThread 9
 java.lang.OutOfMemoryError
     at java.io.ByteArrayOutputStream.expand(ByteArrayOutputStream.java:93)
     at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:218)
     at com.threed.jpct.ZipHelper.unzip(ZipHelper.java:30)
     at com.threed.jpct.GLRenderer.convertTexture(GLRenderer.java:766)
     at com.threed.jpct.GLRenderer.setTextures(GLRenderer.java:2151)
     at com.threed.jpct.GLRenderer.drawVertexArray(GLRenderer.java:2064)
     at com.threed.jpct.World.draw(World.java:1341)
     at com.threed.jpct.World.draw(World.java:1122)
     at gp.itsolutions.Render.onDrawFrame(Render.java:87)
     at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1332)
     at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1116)

I am rewriting and cleaning the code for what I have now in my mmo game (in a separate project)
and get this error when (I think) loading the terrain from zip file (as ZipHelper is mentioned in the error)
But the fact is, I did not change anything compared to my old code, except for the fact that I created a separate class for loading and disposing the terrain...

Terrain class:
Code: [Select]
public class Terrain {
private Object3D terrain = null;
private Texture terrainTexture = null;
private ZipInputStream zis = null;

public Terrain(World world){
try {

if(terrainTexture == null)
terrainTexture = new Texture(Gameplay.resources.openRawResource(R.raw.terraintexture));

if(!TextureManager.getInstance().containsTexture("terrainTexture"))
TextureManager.getInstance().addTexture("terrainTexture", terrainTexture);

if(zis == null){
zis = new ZipInputStream(Gameplay.resources.openRawResource(R.raw.terrain));
zis.getNextEntry();
}

if( (terrain == null) && (zis != null) ){

terrain = Loader.loadSerializedObject(zis);

terrain.setScale(20);
terrain.setTexture("terrainTexture");
//F.tileTexture(terrain, 25);
terrain.compile();
terrain.build();
world.addObject(terrain);

}

} catch (Exception e) {
e.printStackTrace();
}
}

public void dispose(){
if(terrainTexture != null){ terrainTexture = null; }
if(terrain != null){ terrain = null; }
if(zis != null){ try { zis.close(); } catch (IOException e) { } zis = null; }
}
}

I have tried several attempts to try and fix it, but I can't get the right one..
Figured a second pair of eyes might help..
Clean code is simple and direct. Clean code reads like well-written prose. Clean code never obscures the designer’s intent but rather is full of crisp abstractions and straightforward lines of control. - Grady Booch

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Out of memory
« Reply #1 on: February 24, 2011, 03:00:28 pm »
It crashes while converting the texture (how large is it?) not while loading the zip. You can easily spot that in the stack trace.
I've a few hints, but i don't think that they'll cut it:

  • Don't store the zis in the class but move it into the method instead. You don't need it after loading, it just eats some memory for no gain.
  • Try to call strip() on the terrain to free some memory.
  • Try to use 4bpp for the texture by calling enable4bpp(true);
  • Try to compile the terrain using indexed geometry by calling forceGeometryIndices(true);

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Out of memory
« Reply #2 on: February 24, 2011, 03:02:31 pm »
And another one:

  • Load the texture first, add it to the TextureManager, call preWarm() on the TextureManager and then load the model.

Offline Kaiidyn

  • long
  • ***
  • Posts: 103
    • View Profile
Re: Out of memory
« Reply #3 on: February 24, 2011, 04:45:00 pm »
The preWarm on the TextureManager fixed the first problem :)

Code: [Select]
public class Terrain {
private Object3D terrain = null;
private Texture terrainTexture = null;

public Terrain(World world){


if(terrainTexture == null){
terrainTexture = new Texture(Gameplay.resources.openRawResource(R.raw.terraintexture));
terrainTexture.enable4bpp(true);
}
if(!TextureManager.getInstance().containsTexture("terrainTexture"))
TextureManager.getInstance().addTexture("terrainTexture", terrainTexture);

TextureManager.getInstance().preWarm(Gameplay.render.frameBuffer);

ZipInputStream zis = new ZipInputStream(Gameplay.resources.openRawResource(R.raw.terrain));
try {
zis.getNextEntry();
} catch (IOException e) {
throw new RuntimeException(e);
}

if( (this.terrain == null) ){

this.terrain = Loader.loadSerializedObject(zis);
this.terrain.strip();
this.terrain.setScale(20);
this.terrain.setTexture("terrainTexture");
//F.tileTexture(terrain, 25);
this.terrain.forceGeometryIndices(true);
this.terrain.compile();
this.terrain.build();
world.addObject(this.terrain);

}else{
Log.e("Terrain already loaded..");
}

try {
zis.close();
zis = null;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

public void dispose(){
if(terrainTexture != null){ terrainTexture = null; }
if(terrain != null){ terrain = null; }
}
}
Do I still need to call terrain.compile() after terrain.forceGeometryIndices(true) ? (or before?)
And is this the proper way to use enable4bpp, as I assume the texture is already being loaded when initializing the terrainTexture variable, the enable4bpp will need to convert it to 4bpp which in turn will consume more memory (or cpu)

When I call onStop and dispose everything (or nullify) and restart the game, it is giving another out of memory error
Code: [Select]
ERROR/dalvikvm-heap(4283): 288000-byte external allocation too large for this process.
 FATAL EXCEPTION: GLThread 10
 java.lang.OutOfMemoryError
     at org.apache.harmony.luni.platform.OSMemory.malloc(Native Method)
     at org.apache.harmony.luni.platform.PlatformAddressFactory.alloc(PlatformAddressFactory.java:150)
     at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:66)
     at java.nio.ReadWriteDirectByteBuffer.<init>(ReadWriteDirectByteBuffer.java:51)
     at java.nio.BufferFactory.newDirectByteBuffer(BufferFactory.java:93)
     at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:68)
     at com.threed.jpct.CompiledInstance.fill(CompiledInstance.java:698)
     at com.threed.jpct.Object3DCompiler.compile(Object3DCompiler.java:136)
     at com.threed.jpct.World.compile(World.java:2037)
     at com.threed.jpct.World.renderScene(World.java:1087)
     at gp.itsolutions.Render.onDrawFrame(Render.java:86)
     at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1332)
     at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1116)
Not sure what this means, or how to fix it.

Edit: It appears I start my game too fast after I shut it down, when I wait a couple of seconds it does work..  :D
edit2: It happens randomly now...  :-\
« Last Edit: February 24, 2011, 04:59:54 pm by Kaiidyn »
Clean code is simple and direct. Clean code reads like well-written prose. Clean code never obscures the designer’s intent but rather is full of crisp abstractions and straightforward lines of control. - Grady Booch

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Out of memory
« Reply #4 on: February 24, 2011, 05:45:00 pm »
Call forceGeometryIndices() before compile(). Your way to enable 4bpp is fine. The call itself doesn't consume one single byte of memory. The conversion happens during the upload process to the GPU. The texture data in memory is always 8bpp no matter which setting you use.

The second exception is an indication that you are not disposing and nullifying everything as needed. Keep in mind that for example
Code: [Select]
a=new int[1000];
a=new int[1000];

hasn't the same memory footprint as

Code: [Select]
a=new int[1000];
a=null;
a=new int[1000];

Apart from all that, you already seem to live on the edge of memory...you won't get very far with this as the game commences IMHO. What's the polygon count of the terrain, what's the size of the texture? Something seems to be too big here...



Offline Kaiidyn

  • long
  • ***
  • Posts: 103
    • View Profile
Re: Out of memory
« Reply #5 on: February 24, 2011, 06:37:47 pm »
Yea, I was afraid of that....
I am using a 256x256 texture for heightmap and run it through Terragen, a Heightmap2Serialized object generator (based on wiki page Heightmap),
Maybe there are some modifications possible in the terrain generation there.
Then I scale the terrain object times 20 (terrain.setScale(20)).
Also when closing my game I run System.gc() in the Activity.onClose(), and it's required to wait for that to done before running game again without the error.

Maybe it is possible to 'stream' the terrain somehow?
Or only load x*x around the player into the memory, and as the player walks, unload the parts of the terrain that are not visible anyway..
Don't think that is easy to do, if at all possible.
Clean code is simple and direct. Clean code reads like well-written prose. Clean code never obscures the designer’s intent but rather is full of crisp abstractions and straightforward lines of control. - Grady Booch

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Out of memory
« Reply #6 on: February 24, 2011, 09:28:57 pm »
A 256*256 texture for the heightmap...ok, but what exactly does this mean for polygon/vertex count? You might want to have a look at the wiki where paulscode wrote a nice tutorial about how to reduce polygon count with free tools. I did this for the terrain in the An3DBenchXL bench and it worked pretty well. Maybe that's an option...

Offline Kaiidyn

  • long
  • ***
  • Posts: 103
    • View Profile
Re: Out of memory
« Reply #7 on: February 24, 2011, 09:51:23 pm »
I have been looking at that page before and well,
I really don't like blender, that just doesn't work for me at all.. (+ I cant design for shit =p even in 3dsmax7 back in the day)
Also, I am a Linux user, and those programs are for windows.

Maybe an unappropriated question but could I have the terrain that you used in An3DBenchXL? (at least for testing purposes)
Then I could focus more on the 'more important' stuff and add/change terrain, models etc later.
Clean code is simple and direct. Clean code reads like well-written prose. Clean code never obscures the designer’s intent but rather is full of crisp abstractions and straightforward lines of control. - Grady Booch

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Out of memory
« Reply #8 on: February 24, 2011, 09:53:33 pm »
Sure you can have that one. I'll upload it tomorrow now. I still would like to know the actual polygon/vertex count of your terrain to get an idea of what we are talking about here.

Terrain (ser file (named .mp3) and the texture): http://jpct.de/download/tmp/terrain.7z

use for example
Code: [Select]
Camera cam = world.getCamera();
cam.setPosition(17552.824f, -1755.2219f, 9189.418f);
cam.rotateY(-0.58f);
cam.rotateX(-0.17f);

as a starting position.

I'm doing this to initialize it:
Code: [Select]
terrain.forceGeometryIndices(true);
terrain.build();
terrain.strip();
« Last Edit: February 24, 2011, 09:59:05 pm by EgonOlsen »

Offline Kaiidyn

  • long
  • ***
  • Posts: 103
    • View Profile
Re: Out of memory
« Reply #9 on: February 24, 2011, 10:08:05 pm »
Ok, Thanks for the object, really appreciate it.
About the polygon/vertex count.. how would I be able to get that? using the PolygonManager?

Edit: When loading your terrain object, it tells me it's an Unsupported version: 1347093252 and wont load.
« Last Edit: February 24, 2011, 10:13:15 pm by Kaiidyn »
Clean code is simple and direct. Clean code reads like well-written prose. Clean code never obscures the designer’s intent but rather is full of crisp abstractions and straightforward lines of control. - Grady Booch

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Out of memory
« Reply #10 on: February 24, 2011, 10:42:15 pm »
You have unzipped the file?

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Out of memory
« Reply #11 on: February 24, 2011, 10:43:30 pm »
Oh, and you can get both information (vertex/polygon) from the Mesh.

Offline Kaiidyn

  • long
  • ***
  • Posts: 103
    • View Profile
Re: Out of memory
« Reply #12 on: February 24, 2011, 11:23:57 pm »
Yea, I unzipped it renamed the file (back to) .ser, rezipped it to use in the same way as I did before...

Will try to get the poly count tomorrow..
Clean code is simple and direct. Clean code reads like well-written prose. Clean code never obscures the designer’s intent but rather is full of crisp abstractions and straightforward lines of control. - Grady Booch

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Out of memory
« Reply #13 on: February 24, 2011, 11:27:21 pm »
Which version of jPCT-AE are you using? The latest beta from the new version thread?

Offline icarusfactor

  • int
  • **
  • Posts: 58
    • View Profile
Re: Out of memory
« Reply #14 on: February 25, 2011, 09:53:10 am »
I was going to enter a new topic but saw this one on the same issue I am having.

Code: [Select]
E/AndroidRuntime( 2692): FATAL EXCEPTION: GLThread 9
E/AndroidRuntime( 2692): java.lang.OutOfMemoryError
E/AndroidRuntime( 2692):        at com.threed.jpct.Texture.loadTexture(Texture.java:742)
E/AndroidRuntime( 2692):        at com.threed.jpct.Texture.<init>(Texture.java:192)
E/AndroidRuntime( 2692):        at com.threed.jpct.molespect.MoleSpect.loadTextures(MoleSpect.java:1364)
E/AndroidRuntime( 2692):        at com.threed.jpct.molespect.MoleSpect.Text2PosTexture(MoleSpect.java:2022)
E/AndroidRuntime( 2692):        at com.threed.jpct.molespect.MoleSpect.TextFormatTexture(MoleSpect.java:1819)
E/AndroidRuntime( 2692):        at com.threed.jpct.molespect.MoleSpect.showElementpage(MoleSpect.java:1673)
E/AndroidRuntime( 2692):        at com.threed.jpct.molespect.MoleSpect$MyRenderer.onDrawFrame(MoleSpect.java:2364)
E/AndroidRuntime( 2692):        at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1332)
E/AndroidRuntime( 2692):        at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1116)


I tried the preWarm() option but when I compile it says:
 
Code: [Select]
cannot find symbol
    [javac] symbol  : method preWarm(com.threed.jpct.FrameBuffer)
    [javac] location: class com.threed.jpct.TextureManager
    [javac]          TextureManager.getInstance().preWarm(fb);

While I use "fb" as a global for blitting my text which works fine. It is created like this
in my global section.

private FrameBuffer fb = null;

I use "fb" in multiple functions and works in the rest of the program.


Also, on the memory issue I run a memory checker and i tried:

replacing the texture --- memory usage still grows with each use.
unload and remove the texture then load news ones back in -- memory usage still grows with each use.
flush and reload all textures then add my new texture - memory still grows with each use.
Comment out the replacement of the new texture -- works fine memory stays in green, over and over just fine. Just does not add my modification texture. :(

No success with any of these options will keep trying to figure out why preWarm() is not working.