Difference between revisions of "Hybrid GPU Shader Animations for Bones"

From JPCT
Jump to: navigation, search
(The Code)
(The Code)
Line 123: Line 123:
  
 
'''JPCTNamespaceUtils Class:'''
 
'''JPCTNamespaceUtils Class:'''
 +
 
This class must be in the package 'com.threed.jpct'.  This helper class is setup specifically to be able to access namespace restricted variables/methods.
 
This class must be in the package 'com.threed.jpct'.  This helper class is setup specifically to be able to access namespace restricted variables/methods.
 
<pre>
 
<pre>
Line 141: Line 142:
  
 
'''BonesNamespaceUtils Class:'''
 
'''BonesNamespaceUtils Class:'''
 +
 
This class must be in the package 'raft.jpct.bones'.  This helper class is setup specifically to be able to access namespace restricted variables/methods.
 
This class must be in the package 'raft.jpct.bones'.  This helper class is setup specifically to be able to access namespace restricted variables/methods.
 
<pre>
 
<pre>
Line 240: Line 242:
  
 
'''Vertex Shader Snippet:'''
 
'''Vertex Shader Snippet:'''
This class must be in the package 'raft.jpct.bones'.  This helper class is setup specifically to be able to access namespace restricted variables/methods.
 
 
<pre>
 
<pre>
 
...
 
...

Revision as of 05:17, 25 June 2016

Overview

While working on an Android project, I found that jPCT+Bones for skeletal animations were becoming too CPU intensive on the UI thread. When adding twenty animated models on screen, I was noticing a considerable drop in the frame-rate. So I developed a hybrid solution of offloading calculations to the GPU for my project, which I will share with you now.

I have added notes and personal recommendations. This architecture may not be perfect for your project, so please alter and use to fit your needs.

Comments and suggestions are welcome as I will be adding a FAQ to the bottom of the page.

Requirements

JPCT (requires GL ES 2+ as it uses GLSL Shaders) Bones

How It Works

This implementation works by offloading all of the vertex calculations to the GPU. The bone matrix transforms for the poses are still performed on the CPU side.

Gains: Speed. You will harness the GPU which is setup to handle floating point calculation quickly and is dedicated to such tasks. As an example with my project: having 20 animated models dropped the frame-rate to 8-9 fps on my device. Using this method, it went back up to the 60-62 fps cap.

Losses: Software knowledge of the transformed pose. Since the meshes of your models will not be animated on the software-side, it will have no knowledge of its transformed pose. The polygons will be in the initial state, which will not give you an accurate bounding-box, ray-trace collisions, etc... of the current animated mesh pose as it will not actually be transformed on the software-side.

The Code

This code is free, open-source code and you are welcome to use it as you see fit.


GPUAnimated3D Class:

package yournamespacegoeshere;

import yournamespacegoeshere.GPUAnimated3DShader;
import com.threed.jpct.GLSLShader;
import com.threed.jpct.IRenderHook;
import com.threed.jpct.Object3D;

import raft.jpct.bones.Animated3D;
import raft.jpct.bones.BonesNamespaceUtils;
import raft.jpct.bones.SkinClip;

/**
 * Created by Dougie on 6/21/16.
 */
public class GPUAnimated3D extends Animated3D implements IRenderHook {
    public GPUAnimated3D(Animated3D object) {
        super(object);
        BonesNamespaceUtils.setSkinAttributes(object);
        setRenderHook(this);
    }

    public void animateSkin(float index, int sequence) {
        if(getSkinClipSequence() != null) {
            if(sequence == 0) {
                BonesNamespaceUtils.animate(this, index*getSkinClipSequence().getTime(), getSkeletonPose());
            } else {
                SkinClip clip = getSkinClipSequence().getClip(sequence - 1);
                clip.applyTo(index * clip.getTime(), getSkeletonPose());
            }
            getSkeletonPose().updateTransforms();
        }
    }
    public void animatePose(float index, int sequence, float weight) {
        BonesNamespaceUtils.animatePoseDontApply(this, index, sequence, weight);
    }

    @Override
    public void beforeRendering(int i) { ; }
    @Override
    public void afterRendering(int i) { ; }
    @Override
    public void setCurrentObject3D(Object3D object3D) { ; }
    @Override
    public void setCurrentShader(GLSLShader glslShader) {
        if(glslShader!=null && glslShader instanceof GPUAnimated3DShader) {
            ((GPUAnimated3DShader)glslShader).updateBeforeRenderingObject(this);
        }
    }
    @Override
    public void setTransparency(float v) { ; }
    @Override
    public void onDispose() { ; }
    @Override
    public boolean repeatRendering() { return false; }
}


GPUAnimated3DShader Class:

package yournamespacegoeshere;

import android.opengl.GLES20;

import com.threed.jpct.GLSLShader;
import com.threed.jpct.Matrix;

import raft.jpct.bones.Animated3D;
import raft.jpct.bones.BonesNamespaceUtils;

/**
 * Created by Dougie on 6/21/16.
 */
public class GPUAnimated3DShader extends GLSLShader {
    int skinWeightsHandle = -1;
    int jointIndicesHandle = -1;

    public GPUAnimated3DShader(String vertexShaderSource, String fragmentShaderSource) {
        super(vertexShaderSource, fragmentShaderSource);

        skinWeightsHandle = GLES20.glGetAttribLocation(getProgram(), BonesNamespaceUtils.ATTR_SKIN_WEIGHTS);
        jointIndicesHandle = GLES20.glGetAttribLocation(getProgram(), BonesNamespaceUtils.ATTR_JOINT_INDICES);
    }

    public void updateBeforeRenderingObject(Animated3D pObject) {
        if(pObject!=null) {
            Matrix[] skelPose = BonesNamespaceUtils.getSkeletonPosePallete(pObject.getSkeletonPose());
            if(skelPose!=null && skelPose.length>0) {
                setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE_SIZE, skelPose.length);
                setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE, skelPose);
            } else {
                setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE_SIZE, 0);
            }
        }
    }
}


JPCTNamespaceUtils Class:

This class must be in the package 'com.threed.jpct'. This helper class is setup specifically to be able to access namespace restricted variables/methods.

package com.threed.jpct;

/**
 * Created by Dougie on 6/18/16.
 */
public class JPCTNamespaceUtils {
    public static boolean VertexAttributesNameIs(VertexAttributes pVertAttrs, String pName) {
        if(pVertAttrs!=null && pName!=null && pVertAttrs.name!=null && pVertAttrs.name.equals(pName))
            return true;
        return false;
    }
}


BonesNamespaceUtils Class:

This class must be in the package 'raft.jpct.bones'. This helper class is setup specifically to be able to access namespace restricted variables/methods.

package raft.jpct.bones;

import com.threed.jpct.Matrix;
import com.threed.jpct.JPCTNamespaceUtils;
import com.threed.jpct.VertexAttributes;

/**
 * Created by Dougie on 6/17/16.
 */
public class BonesNamespaceUtils {
    public static final String ATTR_SKIN_WEIGHTS = "skinWeights";
    public static final String ATTR_JOINT_INDICES = "jointIndices";
    public static final String UNIFORM_SKEL_POSE = "skelPose";
    public static final String UNIFORM_SKEL_POSE_SIZE = "skelPoseSize";

    public static void animate(Animated3D pAnimated3D, float pTime, SkeletonPose pPose) {
        if(pAnimated3D!=null && pAnimated3D.getSkinClipSequence()!=null) {
            pAnimated3D.getSkinClipSequence().animate(pTime, pPose);
        }
    }
    public static void animatePoseDontApply(Animated3D pAnimated3D, float index, int sequence, float weight) {
        if(pAnimated3D!=null) {
            pAnimated3D.animatePoseDontApply(index, sequence, weight);
        }
    }
    public static Matrix[] getSkeletonPosePallete(SkeletonPose pPose) {
        if(pPose!=null) {
            return pPose.palette;
        }
        return null;
    }
    public static void setSkinAttributes(Animated3D pAnimated3D) {
        int i, j, k, len;
        if(pAnimated3D != null && pAnimated3D.getMesh()!=null) {
            boolean hasWeights = false, hasJointIndices = false;
            VertexAttributes[] vertAttrs = pAnimated3D.getMesh().getVertexAttributes();
            if(vertAttrs!=null) {
                //Loop backwards as they should be the last added ones if shared mesh
                for(i=vertAttrs.length-1;i>=0;--i) {
                    if(JPCTNamespaceUtils.VertexAttributesNameIs(vertAttrs[i], ATTR_SKIN_WEIGHTS)) {
                        hasWeights = true;
                    } else if(JPCTNamespaceUtils.VertexAttributesNameIs(vertAttrs[i], ATTR_JOINT_INDICES)) {
                        hasJointIndices = true;
                    }
                    if(hasWeights && hasJointIndices) {
                        break;
                    }
                }
            }

            if(!hasWeights) {
                float[][] weights = pAnimated3D.skin.weights;
                if(weights != null && weights.length > 0) {
                    len = weights[0].length;
                    float[] weightsArr = new float[pAnimated3D.getMesh().getUniqueVertexCount() * len];
                    for (j = 0; j < weights.length; ++j) {
                        for (k = 0; k < len; ++k) {
                            weightsArr[j * len + k] = weights[j][k];
                        }
                    }
                    if (weights.length < pAnimated3D.getMesh().getUniqueVertexCount()) {
                        for (j = weights.length; j < pAnimated3D.getMesh().getUniqueVertexCount(); ++j) {
                            for (k = 0; k < len; ++k) {
                                weightsArr[j * len + k] = 0f;
                            }
                        }
                    }
                    pAnimated3D.getMesh().addVertexAttributes(new VertexAttributes(ATTR_SKIN_WEIGHTS, weightsArr, len));
                }
            }
            if(!hasJointIndices) {
                short[][] jointIndices = pAnimated3D.skin.jointIndices;
                if (jointIndices != null && jointIndices.length > 0) {
                    len = jointIndices[0].length;
                    float[] jointIndicesArr = new float[pAnimated3D.getMesh().getUniqueVertexCount() * len];
                    for (j = 0; j < jointIndices.length; ++j) {
                        for (k = 0; k < len; ++k) {
                            jointIndicesArr[j * len + k] = jointIndices[j][k];
                        }
                    }
                    if (jointIndices.length < pAnimated3D.getMesh().getUniqueVertexCount()) {
                        for (j = jointIndices.length; j < pAnimated3D.getMesh().getUniqueVertexCount(); ++j) {
                            for (k = 0; k < len; ++k) {
                                jointIndicesArr[j * len + k] = 0f;
                            }
                        }
                    }
                    pAnimated3D.getMesh().addVertexAttributes(new VertexAttributes(ATTR_JOINT_INDICES, jointIndicesArr, len));
                }
            }
        }
    }
}


Vertex Shader Snippet:

...
uniform mat4 modelViewProjectionMatrix;
...
uniform mat4 skelPose[50];
uniform int skelPoseSize;

attribute vec4 position;
attribute vec3 normal;
...
attribute vec4 skinWeights;
attribute vec4 jointIndices;

void main(void)
{
	vec4 animPosition = vec4(0.0,0.0,0.0, position[3]);
	vec3 animNormal = vec3(0.0,0.0,0.0);
	if(skelPoseSize>0) {
		float weight;
		for(int j=0; j<4; ++j) {
			weight = skinWeights[j];
			if(weight != 0.0) {
				mat4 boneMat = skelPose[int(floor(jointIndices[j]+0.5))];
				animPosition.xyz += vec3(position.x*boneMat[0][0] + position.y*boneMat[1][0] + position.z*boneMat[2][0] + boneMat[3][0],
										position.x*boneMat[0][1] + position.y*boneMat[1][1] + position.z*boneMat[2][1] + boneMat[3][1],
										position.x*boneMat[0][2] + position.y*boneMat[1][2] + position.z*boneMat[2][2] + boneMat[3][2])*weight;
				animNormal += vec3(normal.x*boneMat[0][0] + normal.y*boneMat[1][0] + normal.z*boneMat[2][0],
								  normal.x*boneMat[0][1] + normal.y*boneMat[1][1] + normal.z*boneMat[2][1],
								  normal.x*boneMat[0][2] + normal.y*boneMat[1][2] + normal.z*boneMat[2][2])*weight;
			}
		}
		animPosition.yz *= -1.0;
		animNormal.yz *= -1.0;
	} else {
		animPosition = position;
		animNormal = normal;
	}

	...

	gl_Position = modelViewProjectionMatrix * animPosition;
}