jPCT-AE - a 3d engine for Android > Support

Problem with object picking and modified camera backbuffer matrix

(1/5) > >>

redfalcon:
Hi,

I'm currently developing an AR-App and use jPCT-ae for rendering. To affix the model on a marker I receive a pose matrix and apply her to the camera with camera.setBack(). This works for putting my model on the marker, but I can't get picking to work with it. If I just render my model without changing the camera matrix it works fine, but as soon as I change her, it doesn't work anymore (calcMinDistanceAndObject3D() returns everywhere on my object COLLISION_NONE). Since the picking needs the camera position, I assume that something with the position vector is wrong.

My picking code:

--- Code: ---
private void doPicking(int touchX, int touchY){
SimpleVector dir = Interact2D.reproject2D3DWS(camera, frameBuffer, touchX, touchY).normalize();
Object[] res = world.calcMinDistanceAndObject3D(camera.getPosition(), dir, 10000);
  if (res[1] != null) {
    //do something...
 
  }
}

--- End code ---


Object properties:


--- Code: --- Object3D[] serializedObject = Loader.loadSerializedObjectArray(serializedInputStream);

for (int i = 0; i < serializedObject.length; i++) {
serializedObject[i].setCollisionMode(Object3D.COLLISION_CHECK_OTHERS);
if (i > 0) serializedObject[i].addParent(serializedObject[0]);
serializedObject[i].strip();

}


--- End code ---

Camera pose setting:


--- Code: ---
public void onDrawFrame(GL10 arg0) {

//...

float[] matrix = getMatrixForJPCT(new float[16]);

Matrix mat = new Matrix();
mat.setDump(matrix);

//If I comment this out, picking works flawless:
camera.setBack(mat);

//...
}


--- End code ---

Any idea why that is?

EgonOlsen:
Does the tweaked matrix include a translation? If so, try to remove it from the matrix and set it as the camera's position instead.

redfalcon:
Thanks, that brought me on the right track. The AR-engine converted the pose to a OpenGL-compatible matrix beforehand, which additionally screwed my calculations. But now I have some trouble doing the correct conversions between the coordinate systems now.
The AR-engine uses a right handed system (Picture). According to your wiki, I need to rotate the matrix by 180° around x to convert it to the jPCT system. But do I need to do it before or after setting the camera matrix? I currently extract the rotation/translation from the float array, dump it into a Matrix/SimpleVector, rotate the Matrix/SimpleVector by 180° around x and set them as the backbuffer/position. The car is fixed to the marker, but behaves totally unpredictable (it seems as the model sticks to the camera, and not to the marker) as I move around my device, so I guess my conversion is wrong.

EgonOlsen:
Apart from the picking, was it working before? If so, just do what you did before separate the rotation from the translation afterwards.

redfalcon:
Yes, besides some unwanted minor movement of the model when I move my device, everything but picking works. But the separating doesn't really work. As soon as I set the position of the camera, the model doesn't stay on the marker.

The GL-like matrix that does the placement correctly (but not the picking) looks like this:

--- Code: ----0.996   0.069  -0.063   0.000
0.092   0.808  -0.582   0.000
0.011  -0.586  -0.811   0.000
-75.993 -110.593 1902.010   1.000

--- End code ---

I set this directly as the backbuffer and leave the position alone.

The unmodified pose itself is just transposed:


--- Code: ----0.996   0.092   0.011 -75.993
0.069   0.808  -0.586 -110.593
-0.063  -0.582  -0.811 1902.010

--- End code ---

Looks like a 3x3 rotation matrix and a 3x1 translation vector to me.

If I set this matrix as the cameras backbuffer and this vector as the position, the picking works but the model is somewhere far off and doesn't stay on the marker if I move the device.

Navigation

[0] Message Index

[#] Next page