www.jpct.net
Bones - Skeletal and Pose Animations for jPCT/jPCT-AE => Bones => Topic started by: kkl on May 08, 2014, 06:22:10 pm
-
Hi Raft,
Is it doable for Bones to have hardware skinning? I checked from orge3d.org and they seem to have it. http://www.ogre3d.org/forums/viewtopic.php?f=4&t=66907#p440560 (http://www.ogre3d.org/forums/viewtopic.php?f=4&t=66907#p440560)
I have 6 animated models with bones and it seems to takes a lot of CPU usage and slow down Android main UI (I'm doing live wallpaper).
-
yes, it is possible but i dont have any plans or time to implement it. sorry
-
Is it possible if we do it without editing the Bones library? I'll try to implement it if no editing required.
-
no, you need to modify Bones' source. but modification will not be that much. i'm not completely sure about this but I suppose, you should leave calculation of skeleton poses as it is and intervene at applyAnimation method. you should upload poses to GPU and calculate new positions of vertices via shaders.
if you intend to do it, jME2's relevant part may help. it's included in Bones distribution.
also note, with hw skinning, jPCT's polygon level collision detection will not work correctly (unless of course you download vertex positions from GPU). but you can still use sphere or ellipsoid collision detection.
-
you should leave calculation of skeleton poses as it is and intervene at applyAnimation method
Does it mean animateSkin() method is to be processed by CPU and applyAnimation() by GPU? Can we put the animateSkin() calculation to shader as well? Hoping we could just put all calculations to GPU so CPU would process as less as possible.
jME2's relevant part may help. it's included in Bones distribution
It seems like jME2 is a really huge library, but it's open-sourced. I guess I can try to find the needle in the ocean ;P
-
Does it mean animateSkin() method is to be processed by CPU and applyAnimation() by GPU?
yes, that is what I meant
Can we put the animateSkin() calculation to shader as well? Hoping we could just put all calculations to GPU so CPU would process as less as possible.
that is also possible but it requires much more modification to Bones' code. calculation of skeleton pose is relatively cheaper. the expensive part is applying skeleton pose to mesh. I will suggest starting with porting applyAnimation() to shader. this will have a couple of benefits:
* it wil be easier to implement
* if something does not work, you can be sure the problem is in applyAnimation
It seems like jME2 is a really huge library, but it's open-sourced. I guess I can try to find the needle in the ocean ;P
yes, jME2 is a huge library but Bones comes with only relevant parts. particularly have a look at com.jmex.model.ogrexml.anim.MeshAnimationController update method
-
If you manage to do the skinning on the GPU, it would be great if the required shader code is designed as a kind of plugin, so that it's possible to inject it into other shaders by doing some simple insert operation. Because otherwise, it would require everybody who is going to use this to write and manage his/her own shaders and the default shaders would be useless.
-
Hi Raft,
Yea. I think I can start from applyAnimation(). BTW, I checked by calculating time taken for animateSkin() and applyAnimation() respectively, and found out the time taken for animateSkin() is higher than applyAnimation(). The result is, animateSkin() = ~450000 ns and applyAnimation() = ~45000 ns in average. Since it's taking more process in animateSkin(), just hope we can put both in shader too. But I guess that will deeply involve library editing.
-
Hi Egon,
Yea. I guess the hardware skinning is somewhat important. I'll try to implement it. If it works, I'll let y'all know.
-
Yea. I think I can start from applyAnimation(). BTW, I checked by calculating time taken for animateSkin() and applyAnimation() respectively, and found out the time taken for animateSkin() is higher than applyAnimation(). The result is, animateSkin() = ~450000 ns and applyAnimation() = ~45000 ns in average. Since it's taking more process in animateSkin(), just hope we can put both in shader too. But I guess that will deeply involve library editing.
oops, sorry, skin animation is actually applied at applySkeletonPose method, applyAnimation only says jPCT new mesh is ready to upload to GPU.
animateSkin also calls applySkeletonPose so the time you saw is combined time of both
-
In this case, does it mean we have to rewrite the whole animateSkin() to shader? Still doable w/o editing the library?
-
yes, almost the same thing. have a look at the code, you will see it.
-
Ok, I'll have a look at it and keep you guys posted ;)
-
I started reading the code and I just wonder if I can access SkinClipSequence animate() by making it a public method, so I can separate recode the animateSkin and take the applySkeletonPose() to shader and leave the rest to CPU.
-
go ahead, try it
-
I can't use SkinClipSequence animate() method as it's a private method. Could you compile a new Bones.jar with the method declared as PUBLIC? This's the best I can think of. Would love to hear if there is better suggestion ;)
-
you have the source code, just change it and recompile. the zip even contains the ant script to build the whole project ;)
-
Ah~ I missed out the source code in the zip file. I always thought it's close sourced. Ok, i can start working on it now. Thanks alot.
-
Hi Raft,
I've spending few weeks to implemet hardware skinning, but I encountered few problems and I hope you could help me out on this
I've made these code, but the result seems incorrect:
vertex shader: skinPalette is hardcoded to match the bones file I have (im not sure if the number of matrix palettes is fixed).
uniform mat4 modelViewMatrix;
uniform mat4 modelViewProjectionMatrix;
uniform mat4 textureMatrix;
uniform vec4 additionalColor;
uniform vec4 ambientColor;
uniform float alpha;
uniform float shininess;
uniform bool useColors;
uniform float fogStart;
uniform float fogEnd;
uniform vec3 fogColor;
uniform int lightCount;
uniform mat4 skinPalette[7];
uniform vec3 lightPositions[8];
uniform vec3 diffuseColors[8];
uniform vec3 specularColors[8];
uniform float attenuation[8];
attribute vec4 position;
attribute vec3 normal;
attribute vec4 color;
attribute vec2 texture0;
attribute vec2 texture1;
attribute vec2 texture2;
attribute vec2 texture3;
attribute vec4 skinWeights;
attribute vec4 skinJointIndices;
varying vec2 texCoord[4];
varying vec4 vertexColor;
varying vec3 fogVertexColor;
varying float fogWeight;
const vec4 WHITE = vec4(1,1,1,1);
void main() {
texCoord[0] = texture0;
texCoord[1] = (textureMatrix * vec4(texture1, 0, 1)).xy;
texCoord[2] = texture2;
texCoord[3] = texture3;
vec4 myPosition = vec4(position);
vec3 myNormal = vec3(normal);
vec4 vertexTemp;
vec3 normalTemp;
mat4 mat;
for (int i = 0; i < 4; i++) {
mat = skinPalette[int(skinJointIndices[i])];
vertexTemp = mat * position;
vertexTemp *= skinWeights[i];
myPosition += vertexTemp;
normalTemp = normal * mat3(mat);
normalTemp *= skinWeights[i];
myNormal += normalTemp;
}
vec4 vertexPos = modelViewMatrix * myPosition;
vertexColor = ambientColor + additionalColor;
if (lightCount>0) {
// This is correct only if the modelview matrix is orthogonal. In jPCT-AE, it always is...unless you fiddle around with it.
vec3 normalEye = normalize(modelViewMatrix * vec4(myNormal, 0.0)).xyz;
float angle = dot(normalEye, normalize(lightPositions[0] - vertexPos.xyz));
if (angle > 0.0) {
vertexColor += vec4((diffuseColors[0] * angle + specularColors[0] * pow(angle, shininess))*(1.0/(1.0+length(lightPositions[0] - vertexPos.xyz)*attenuation[0])), 1);
}
// Freaky Adreno shader compiler can't handle loops without locking or creating garbage results....this is why the
// loop has been unrolled here. It's faster this way on PowerVR SGX540 too, even if PVRUniSCoEditor says otherwise...
if (lightCount>1) {
angle = dot(normalEye, normalize(lightPositions[1] - vertexPos.xyz));
if (angle > 0.0) {
vertexColor += vec4((diffuseColors[1] * angle + specularColors[1] * pow(angle, shininess))*(1.0/(1.0+length(lightPositions[1] - vertexPos.xyz)*attenuation[1])), 1);
}
}
}
if (fogStart != -1.0) {
fogWeight = clamp((-vertexPos.z - fogStart) / (fogEnd - fogStart), 0.0, 1.0);
fogVertexColor = fogColor * fogWeight;
} else {
fogWeight = -1.0;
}
vertexColor=vec4(min(WHITE, vertexColor).xyz, alpha);
if (useColors) {
vertexColor *= color;
}
gl_Position = modelViewProjectionMatrix * myPosition;
}
This's the code to add vertex attributes to shader. linearizeArray is converting float[][] to float[] and linearizeToFloatArray is short[][] to float[]
float[] f1 = SkinHelper.linearizeArray(skin.weights);
float[] f2 = SkinHelper.linearizeToFloatArray(skin.jointIndices);
skinWeights = new VertexAttributes(
"skinWeights", f1, VertexAttributes.TYPE_FOUR_FLOATS);
skinJointIndices = new VertexAttributes(
"skinJointIndices", f2, VertexAttributes.TYPE_FOUR_FLOATS);
Mesh mesh = getMesh();
mesh.addVertexAttributes(skinWeights);
mesh.addVertexAttributes(skinJointIndices);
This's where I set the uniform for matrix palette (in array form, Egon created a new GLSLShader to pass matrix array).
object3D.setRenderHook(new RenderHookAdapter(){
@Override
public void setCurrentShader(GLSLShader shader) {
super.setCurrentShader(shader);
shader.setUniform("skinPalette", object3D.getSkeletonPose().getPalette());
}
});
I tried to debug glsl and most debugger I tried were not working. Hope you could help me out on this. May be I missed something.
-
I got it worked!! I fixed the previous code with these:
uniform mat4 modelViewMatrix;
uniform mat4 modelViewProjectionMatrix;
uniform mat4 textureMatrix;
uniform vec4 additionalColor;
uniform vec4 ambientColor;
uniform float alpha;
uniform float shininess;
uniform bool useColors;
uniform float fogStart;
uniform float fogEnd;
uniform vec3 fogColor;
uniform int lightCount;
uniform mat4 skinPalette[7];
uniform vec3 lightPositions[8];
uniform vec3 diffuseColors[8];
uniform vec3 specularColors[8];
uniform float attenuation[8];
attribute vec4 position;
attribute vec3 normal;
attribute vec4 color;
attribute vec2 texture0;
attribute vec2 texture1;
attribute vec2 texture2;
attribute vec2 texture3;
attribute vec4 skinWeights;
attribute vec4 skinJointIndices;
varying vec2 texCoord[4];
varying vec4 vertexColor;
varying vec3 fogVertexColor;
varying float fogWeight;
const vec4 WHITE = vec4(1,1,1,1);
void main() {
texCoord[0] = texture0;
texCoord[1] = (textureMatrix * vec4(texture1, 0, 1)).xy;
texCoord[2] = texture2;
texCoord[3] = texture3;
vec4 myPosition = vec4(0,0,0,0);
vec3 myNormal = vec3(0,0,0);
vec4 vertexTemp;
vec3 normalTemp;
mat4 mat;
for (int i = 0; i < 4; i++) {
mat = skinPalette[int(skinJointIndices[i])];
vertexTemp = mat * position;
vertexTemp *= skinWeights[i];
myPosition += vertexTemp;
normalTemp = mat3(mat) * normal;
normalTemp *= skinWeights[i];
myNormal += normalTemp;
}
vec4 vertexPos = modelViewMatrix * myPosition;
vertexColor = ambientColor + additionalColor;
if (lightCount>0) {
// This is correct only if the modelview matrix is orthogonal. In jPCT-AE, it always is...unless you fiddle around with it.
vec3 normalEye = normalize(modelViewMatrix * vec4(myNormal, 0.0)).xyz;
float angle = dot(normalEye, normalize(lightPositions[0] - vertexPos.xyz));
if (angle > 0.0) {
vertexColor += vec4((diffuseColors[0] * angle + specularColors[0] * pow(angle, shininess))*(1.0/(1.0+length(lightPositions[0] - vertexPos.xyz)*attenuation[0])), 1);
}
// Freaky Adreno shader compiler can't handle loops without locking or creating garbage results....this is why the
// loop has been unrolled here. It's faster this way on PowerVR SGX540 too, even if PVRUniSCoEditor says otherwise...
if (lightCount>1) {
angle = dot(normalEye, normalize(lightPositions[1] - vertexPos.xyz));
if (angle > 0.0) {
vertexColor += vec4((diffuseColors[1] * angle + specularColors[1] * pow(angle, shininess))*(1.0/(1.0+length(lightPositions[1] - vertexPos.xyz)*attenuation[1])), 1);
}
}
}
if (fogStart != -1.0) {
fogWeight = clamp((-vertexPos.z - fogStart) / (fogEnd - fogStart), 0.0, 1.0);
fogVertexColor = fogColor * fogWeight;
} else {
fogWeight = -1.0;
}
vertexColor=vec4(min(WHITE, vertexColor).xyz, alpha);
if (useColors) {
vertexColor *= color;
}
gl_Position = modelViewProjectionMatrix * myPosition;
}
where myPosition and myNormal are initialized to zero. But the normal seems to be abit off. I'm looking into it.
-
cool, i'm glad you solved it :)
-
And how much faster is it now?
-
And how much faster is it now?
I tested it out by just doing a simple check on the time taken for the entire drawframe method. Test case is a model w/o texture with ~1968 vertices and 7 bones. Only 3 bones are moving. Test device is Samsung Galaxy S4 with octa core.
Here's the result:
Software: ~ 11-14 ms
Hardware skinning: ~ 1-4 ms
The test result seems to be quite promising. Anyway the rotation seems to be incorrect compared to the software version. I'll try check it out.
-
The test result seems to be quite promising.
really they are :)
-
Cool~ Would you like to share it when you done? This will give others(include me ;) ) a lot of help.
-
Hi Kiffa,
It's almost done, but there's still some problem wif the initial rotation and I didn't have time to fix that yet. It would be great if you could fix it.
The attachment contains the required java files. You would need to download the Bones source code and replace the java files with the same names.
Example usage:
try {
group = BonesIO.loadGroup(context.getAssets().open(
"models/monkey.bones"));
} catch (Exception ex) {
ex.printStackTrace();
}
for (final Animated3D o : group) {
try {
final GLSLShader shader1 = new GLSLShader(
Loader.loadTextFile(context.getAssets().open("shaders/vertex_hardware_skinning.src")),
Loader.loadTextFile(context.getAssets().open("shaders/fragment.src")));
o.setShader(shader1);
o.setRenderHook(new RenderHookAdapter(){
@Override
public void beforeRendering(int arg0) {
shader1.setUniform("skinPalette", o.getSkeletonPose().getPalette());
}
});
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
o.setHardwareEnabled(true);
o.build();
o.discardMeshData();
}
Currently, I hard coded the vertex shader to support up to 10 bones only. If you wanna increase it, edit skinPalette[10] to bigger size array in vertex_hardware_skinning.src. The vertex shader also supports up to 2 light sources only, but you can go ahead and copy part of jpct original vertex shader and paste it to increase light sources.
[attachment deleted by admin]
-
Hi raft and egon,
Sorry for bringing this up. Previously I have implemented hardware skinning and was having initial rotation issue, in which the object appears different rotation than the one original with software skinning. The incorrect rotation started to cause issue when JPCT detects clipping boundary and object is not in the screen and it's automatically not showing the object, but in actual it should show as it's just having different rotation after processing in shader, which is after clipping process.
I was just wondering if there's any addition rotations applied before vertex is passed to shader. I'm afraid I might have missed some rotation matrices, or it could be something else that cause the issue. Thanks.
-
No, actually not. All vertex transformations happen inside the shader.
-
Not sure where went wrong.. Currently i cant find the root cause, is it possbile if we can have an option to disable clipping object process for temporary solution?
-
Strangely, I tried negating final position y and z in shader (which i think it's same with rotateX) and it works ok now. I wonder if I missed somewhere in software part.