Author Topic: Transparency lost after deserialization  (Read 3403 times)

Offline Promyclon

  • byte
  • *
  • Posts: 10
    • View Profile
Transparency lost after deserialization
« on: August 21, 2017, 10:52:25 am »
Hi,

I used the DeSerializer to speed-up loading of the model I had in OBJ:

Code: [Select]
class SerializeEngine {
private Object3D[] objects;
private String sourceModelFileName;
private String sourceMaterialFileName;

SerializeEngine(final String obj, final String mtl) {
sourceModelFileName = obj;
sourceMaterialFileName = mtl;
}

int loadObj() throws IOException {
try (InputStream modelFile = new FileInputStream(sourceModelFileName);
InputStream materialsFile = new FileInputStream(sourceMaterialFileName)) {
objects = Loader.loadOBJ(modelFile, materialsFile, 1.0f);
for (Object3D o : objects)
o.build();
}
return objects.length;
}

String serialize() throws FileNotFoundException, IOException {
if (null == objects) throw new NullPointerException("Object not loaded");
if (0 == objects.length) throw new IndexOutOfBoundsException("Object is empty");
int l = sourceModelFileName.lastIndexOf('.');
String destModelFileName = ((l <= 0) ? sourceModelFileName : sourceModelFileName.substring(0, l)) + ".j3d";
try (OutputStream dest = new FileOutputStream(destModelFileName)) {
(new DeSerializer()).serializeArray(objects, dest, true);
}
return destModelFileName;
}
}

Then, instead of slow loading from .obj:

Code: [Select]
elements = Loader.loadOBJ(modelStream, materialStream, 1.0f);
I loaded the model in Android from a serialized form:

Code: [Select]
elements = Loader.loadSerializedObjectArray(modelStream);
and got the model loaded in application in jus about one second. The dispalyed model looked fine except for transparent elements. They lost their transparency. Did I do something wrong or the transparency isn't written to/read from the serialized form? I was looking at the jPCT-AE DeSerializer's code and couldn't find any code related with transparency but It's hard to believe this information is just lost.

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Transparency lost after deserialization
« Reply #1 on: August 21, 2017, 11:59:21 am »
No, that information isn't a part of the serialized data. The idea of the serialized format is to handle the actual data the defines the object, not the attributes of an object.

Offline Promyclon

  • byte
  • *
  • Posts: 10
    • View Profile
Re: Transparency lost after deserialization
« Reply #2 on: August 21, 2017, 12:52:35 pm »
When I load object florm .obj, the attributes are taken from .mtl, assigned to objects by material names when the parser finds o or g lines. The material name is 'forgotten' but material's attributes are kept in object. When serializing some of them, like texture names are exported but some are not. If the object is not bound to a material anymore, how can I retrieve it's attributes?

The point is, I have a nice looking model in .obj & .mtl and I want it look nicely when I load it, faster, from a serialized form. And I cannot rewrite code whenever I change a model. For now I have retrieved from .mtl transparency data like that:

Code: [Select]
Map<String, Float> transMap = new TreeMap<>(); // <MaterialName, d attribute>
transMap.put("glass", 0.34f);
transMap.put("windscreen", 0.44f);
transMap.put("lights", 0.52f);

Only because I split earlier all objects to face groups by material, adding material to group names (because transparency is assigned by object an I had multimaterial objects) I could strip the object name from the original name and _jPCTnn suffix and reassign transparency after importing serialized model.

Code: [Select]
for (Object3D element : elements) {
  String elementMaterial = extractMaterialName(element.getName());
  Float transparency = transMap.get(elementMaterial);
  if (null != transparency) element.setTransparency((int) (100f * transparency)); // is this formula correct?
}

This, however, is an workaround that I cannot use in production because each model would need some implementation.

Of course, I can add all needed material attributes to object names (is there a limitation on object length?) but if we went to binary serialization to avoid text parsing overhead, it's not the direction we'd like to go.

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Transparency lost after deserialization
« Reply #3 on: August 21, 2017, 02:31:16 pm »
I guess I didn't consider it to be important. I had to make a cut somewhere and transparency didn't make it into the format. I'll look into it. Should be easy to add it.

Offline Promyclon

  • byte
  • *
  • Posts: 10
    • View Profile
Re: Transparency lost after deserialization
« Reply #4 on: August 21, 2017, 03:31:05 pm »
Thank you, Egon! Material data is important and so small, compared to the geometry, that adding it cannot harm :)

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Transparency lost after deserialization
« Reply #5 on: August 21, 2017, 07:44:51 pm »
Please try these jars:

http://jpct.de/download/beta/jpct.jar
http://jpct.de/download/beta/jpct_ae.jar

They should add support for transparency to the (De-)Serializer as well as a new Texture constructor that leaves the stream open if wanted.

Offline Promyclon

  • byte
  • *
  • Posts: 10
    • View Profile
Re: Transparency lost after deserialization
« Reply #6 on: August 22, 2017, 09:54:06 am »
Hi Egon,

I appreciate so quick update. I've used new jpct.jar (429 037 bytes) in my Java transcoder and got serialized (.j3d) file just one byte bigger, having stored 166 object in this file.
I copied jpcr_ae.jar (388 667 bytes) into my Android project, invalidated Android Studio caches, rebuilt a project and tried to load my new .j3d file having the following message:
Quote
08-22 06:26:29.204 4388-4411/gov.nasa.iss E/jPCT-AE: [ 1503383189204 ] - ERROR: Can't deserialize object: float[] array expected (410)!
So, something went wrong and I have now only programatically generated surface in my scene.

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Transparency lost after deserialization
« Reply #7 on: August 22, 2017, 10:28:36 am »
I'll look into it..

Offline Promyclon

  • byte
  • *
  • Posts: 10
    • View Profile
Re: Transparency lost after deserialization
« Reply #8 on: August 22, 2017, 12:25:50 pm »
Further test showed that the new jpct_ae.jar successfully loads the old .j3d.

I've also checked the new constructor
Code: [Select]
Texture(InputStream is, boolean useAlpha, boolean closeStream)
After reading the first texture the ZipInputStream was closed. I've found that the chain of passing closeStream parameter is broken here:
Code: [Select]
private void loadTexture(InputStream is, Bitmap img, boolean alpha, boolean closeStream) {
        this.isLoaded = false;
        Logger.log("Loading Texture...", 2);

        try {
            Bitmap image = img;
            boolean recycleMe = false;
            if(img == null) {
                image = BitmapHelper.loadImage(is); // closeStream defaults to true there :(
                recycleMe = true;
            }

And, finally, my mistake: new .j3d has 16 366 092 bytes while the old is 16 364 764 bytes long, that's 1328 bytes more in new file (I took 1kB for 1B), 8 bytes more per each object.

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Transparency lost after deserialization
« Reply #9 on: August 22, 2017, 02:11:51 pm »
Oops...I've obviously forgotten to take the parameter into account. I'll fix that later.

Offline EgonOlsen

  • Administrator
  • quad
  • *****
  • Posts: 12295
    • View Profile
    • http://www.jpct.net
Re: Transparency lost after deserialization
« Reply #10 on: August 22, 2017, 08:55:56 pm »
Please re-download and try again...

Offline Promyclon

  • byte
  • *
  • Posts: 10
    • View Profile
Re: Transparency lost after deserialization
« Reply #11 on: August 28, 2017, 09:02:09 am »
I've re-exported the model, removed workarounds, launched the app and the scene still looks good. Thanks Egon, your betas work!