jPCT-AE - a 3d engine for Android > Support

Problem with object picking and modified camera backbuffer matrix

(1/5) > >>


I'm currently developing an AR-App and use jPCT-ae for rendering. To affix the model on a marker I receive a pose matrix and apply her to the camera with camera.setBack(). This works for putting my model on the marker, but I can't get picking to work with it. If I just render my model without changing the camera matrix it works fine, but as soon as I change her, it doesn't work anymore (calcMinDistanceAndObject3D() returns everywhere on my object COLLISION_NONE). Since the picking needs the camera position, I assume that something with the position vector is wrong.

My picking code:

--- Code: ---
private void doPicking(int touchX, int touchY){
SimpleVector dir = Interact2D.reproject2D3DWS(camera, frameBuffer, touchX, touchY).normalize();
Object[] res = world.calcMinDistanceAndObject3D(camera.getPosition(), dir, 10000);
  if (res[1] != null) {
    //do something...

--- End code ---

Object properties:

--- Code: --- Object3D[] serializedObject = Loader.loadSerializedObjectArray(serializedInputStream);

for (int i = 0; i < serializedObject.length; i++) {
if (i > 0) serializedObject[i].addParent(serializedObject[0]);


--- End code ---

Camera pose setting:

--- Code: ---
public void onDrawFrame(GL10 arg0) {


float[] matrix = getMatrixForJPCT(new float[16]);

Matrix mat = new Matrix();

//If I comment this out, picking works flawless:


--- End code ---

Any idea why that is?

Does the tweaked matrix include a translation? If so, try to remove it from the matrix and set it as the camera's position instead.

Thanks, that brought me on the right track. The AR-engine converted the pose to a OpenGL-compatible matrix beforehand, which additionally screwed my calculations. But now I have some trouble doing the correct conversions between the coordinate systems now.
The AR-engine uses a right handed system (Picture). According to your wiki, I need to rotate the matrix by 180 around x to convert it to the jPCT system. But do I need to do it before or after setting the camera matrix? I currently extract the rotation/translation from the float array, dump it into a Matrix/SimpleVector, rotate the Matrix/SimpleVector by 180 around x and set them as the backbuffer/position. The car is fixed to the marker, but behaves totally unpredictable (it seems as the model sticks to the camera, and not to the marker) as I move around my device, so I guess my conversion is wrong.

Apart from the picking, was it working before? If so, just do what you did before separate the rotation from the translation afterwards.

Yes, besides some unwanted minor movement of the model when I move my device, everything but picking works. But the separating doesn't really work. As soon as I set the position of the camera, the model doesn't stay on the marker.

The GL-like matrix that does the placement correctly (but not the picking) looks like this:

--- Code: ----0.996   0.069  -0.063   0.000
0.092   0.808  -0.582   0.000
0.011  -0.586  -0.811   0.000
-75.993 -110.593 1902.010   1.000

--- End code ---

I set this directly as the backbuffer and leave the position alone.

The unmodified pose itself is just transposed:

--- Code: ----0.996   0.092   0.011 -75.993
0.069   0.808  -0.586 -110.593
-0.063  -0.582  -0.811 1902.010

--- End code ---

Looks like a 3x3 rotation matrix and a 3x1 translation vector to me.

If I set this matrix as the cameras backbuffer and this vector as the position, the picking works but the model is somewhere far off and doesn't stay on the marker if I move the device.


[0] Message Index

[#] Next page

Go to full version