Thank you for the quick response and all the suggestions. My results are below. I noted your suggestions, and then the results.

I commented out:
<code>
jpctWorld.renderScene( frameBuffer );
jpctWorld.draw( frameBuffer );
</code>
and still have the same issue. But I’ll keep this commented out when the shadow helper is enabled since you mentioned this is done in the shadow helper (thanks for the suggestion).

ATTEMPT 1) Suggestion - Try to lower batch size from 4000 to 2000 to 1000 to...and see if that helps.
2000 gives the same issue.

Result:
I tried 2000, 1000, 500, 100. The lower the value the less the crash happens, especially when the number is set to 100. At 100 I have not been able to cause the crash (yet).

Other observations (when batch size was set to 100).
 I noticed that when I set the max shadow map to 8192 the memory of my java process is 1.6GB of RAM at startup, and then eventually after a minute falls down to 910MB, a huge number. When I disable the shadows completely this drops to 380MB. That is a lot of additional memory to display shadows. Is this typical?

 I noticed that the memory of my java process is 760MB of RAM when I set the max shadow map to 1024 instead of 8192 at startup, and drops to 680MB after a minute.



ATTEMPT 2) Suggestion -  compile(...) with preferDisplayLists set to false. This will be slower but it would be interesting to see if it actually works.
Results: I set the batch size to 10000 to force a consistent JVM crash before applying this change (so I can see if this change alone fixes the issue).

After making this change I did not see the crash (even when I set the batch size to 100). However, like you said, there was a hit in performance. My FPS dropped significantly, to the point my application is too slow.

ATTEMPT 3) Suggestion - Set Config.glUseFBO to false at application startup to see if that helps. Will be slower and limited in shadow map resolution, but again it would be interesting to see if it works that way.
Results: I set the batch size to 10000 to force a consistent JVM crash before applying this change (so I can see if this change alone fixes the issue).

When I made the change I got the same JVM crash consistently.


ATTEMPT 4) Suggestion - Try to synchronize your render loop to the object returned by FrameBuffer.getLock();

I grabbed the lock in the render thread, and ran all the framebuffer calls in a synchronized block as suggested. I really try to avoid doing this, since often it kills performance.

Result: Unfortunately, synchronizing all the frame buffer calls does not work and the exception is the same. The crash still happens after waiting for all the framebuffer code is finished. Here is the output:

<code>
AWT-EventQueue-1 - EVWorld - FPS: 0
AWT-EventQueue-1 - EVWorld - paint() - Entering synchronized block.
AWT-EventQueue-1 - EVWorld - paint() - In synchronized block.
AWT-EventQueue-1 - EVWorld - paint() - Left synchronized block ...
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000010815ea9, pid=10328, tid=9472
# …
</code>


<code>
j  org.lwjgl.opengl.GL11.nglDrawElements(IIILjava/nio/Buffer;IJ)V+0
j  org.lwjgl.opengl.GL11.glDrawElements(ILjava/nio/IntBuffer;)V+37
j  com.threed.jpct.CompiledInstance.compileToDL()V+139
j  com.threed.jpct.GLBase.compileDLs()V+57
j  com.threed.jpct.AWTGLRenderer.drawVertexArray(Lcom/threed/jpct/AWTDisplayList;I)V+9
j  com.threed.jpct.AWTJPCTCanvas.paintGL()V+149
j  org.lwjgl.opengl.AWTGLCanvas.paint(Ljava/awt/Graphics;)V+165
</code>

ATTEMPT 5) Suggestion - Try to render the first frame without using shadow mapping, i.e. by using the normal renderScene...draw...display-sequence instead of the ShadowHelper.

There are two ways I tried this. First, I still create a new shadowHelper, but I delay calling updateShadowMap() and drawScene() on the shadow helper until after 5 canvas paints. This fails, because the JVM crash is not caused by updateShadowMap() and drawScene() – in fact if I comment them out the crash still happens. So long as a create a new shadow mapper the crash will happen.

The second way is to wait 5 paint cycles before creating a new shadowHelper. The problem with this approach is once the regular rendering sequence is done, and then you later try to create a new shadowHelper, the program freezes indefinitely when trying to create a new Shadow Helper.

ATTEMPT 6) If nothing else helps, you might want to give this jar a try: http://www.jpct.net/download/beta/jpct.jar It ports jPCT-AE's VBO support back to jPCT. You can try to use them instead of display lists by setting Config.glUseVBO=true; before creating the frame buffer.

Result:
The JVM crash did not happen when I set Config.glUseVBO=true.


Conclusion:
Attempt 2 slows things down too much. Attempt 1 forces me to use a small batch size.  Attempt 6 works, does not seem to restrict the batch size, and I don’t see a major impact on performance. Attempt 6 adds a little time to startup (to compile VBOs), but this is minor.

Now I have two more questions
- Are there any things I should be aware of when using VBOs?
- Would you kindly suggest ways of reducing memory when using shadows (other than reducing the max shadow map size)? Even for small shadow size values, my memory jumps an additional 100 percent when shadows are enabled.

Thanks again.

Thanks for the response and the explanation.

As for applet security, once I signed the jpct jar file in addition to my own Jars the reflection operation was permitted and the rollback happened as I was hoping (to revert from AA mode to normal HW mode). I suppose the reason that opengl was permitted and reflection was not is because opengl was run under a security context that was initiated from my Jar which is signed and thus trusted. Also, LWJGL jars come pre-signed and I guess for development this works fine (for prod all jars should be signed with the same cert). However, the reflection call must have been running under a different security context that was not trusted, but signing the jpct jar file established the trust that was required.

As for the callback, I no longer need it for this particular scenario since it is working as desired. That being said, I think if you did support a callback for other cases where exceptions occur that would be a nice addition. If you wanted to implement a callback I suppose one way is to use injection similar to the way Spring does. In this case, the client application would inject its own exception handler into jPCT that would implement an interface that jPCT understands (i.e. the interface could have an “exceptionOccured” if you want to be consistent or “handleException” method). When an exeption occurs, jPCT could test whether the client assigned (injected) a value to the handleException instance, and, if it did, then jPCT would call the method on that instance. Right now I see jPCT already has a mode that lets the client specify what should happen on an error (such as continue on or abort) which is nice, but allowing the client to handle exceptions the way they want (such as to display a more informative error web page, etc.) would be helpful. This way if an exception that can’t be handled by jPCT occurs, the client will be able to display an error page (or do whatever else) that makes sense in their situation. Otherwise the applet might just go blank if jPCT aborts which is not desirable.

Just a suggestion, I know you probably have bigger fish to fry.

And, thanks again for your help.

Hi,

I noticed that the jPCT exception handler, specifically in the “exceptionOccurred” handler of AWTJPCTCanvas, is triggering a “java.lang.reflect.ReflectPermission suppressAccessChecks” exception when a LWJGLException exception is thrown since I am running an applet in an environment that does not have the reflection permission set.

The problem is that the jPCT “exceptionOccurred” handler is eating the original LWJGLException, and hence the only exception I see is the secondary ReflectPermission (i.e. the only message I see is: ERROR: Unable to recover: access denied (java.lang.reflect.ReflectPermission suppressAccessChecks). Perhaps this handler is trying to do something with the LWJGL exception, and is using reflection in one way or another.

Is there any way to avoid using reflection in the exceptionOccurred method of AWTJPCTCanvas for those environments where reflection is not permitted?

If this is not possible, can you please print the stack trace of the original LWJGLException so that I know what the root cause is? Note: that the case is when I see the jpct message: "Unable to recover: " -- you may print out the LWJGLException in other cases but without the source I can't say for certain.

By the way, I did find the root cause by doing thread dumps, but this was a hit and miss approach. The issue is that my graphics driver was throwing a opengl.WindowsPeerInfo.nChoosePixelFormat related error since I was using a HW framebuffer mode that is not supported by the default drivers.

Thank you! 
Dave