Yes, that causes it. From the docs for setRotationMatrix():
Sets the rotation matrix for the object. Usually, this is not required as long as the rotateX/Y/Z() methods are satisfying your needs. If you do this, make sure to reset the scale by calling setScale(1f) is you previously modified it.
So i suggest to do something like setScale(1f);setRotationMatrix(...);setScale(yourScale);