www.jpct.net

jPCT-AE - a 3d engine for Android => Support => Topic started by: SonicBurst2 on April 29, 2017, 02:02:32 pm

Title: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on April 29, 2017, 02:02:32 pm
Hi,
     I am trying to experiment with MultiPass Rendering with jpct ae . I was trying to do 2 pass Gaussian blur . I was able to get individual blurs Successfully , i.e horizontal and vertical blurs individually . But what I want is ->
1) Render the "Bright" part to a texture.
1) the output from the 1st step should undergo horizontal blurring .
2) the output from the 2nd step should undergo vertical blurring
3) The output from the 3rd step would be blended in some way with the original rendered image for the post processing bloom effect .

   So how do we do multipass rendering in jpct ae ? And pass the result from one render phase to the other for further processing? Also I have to composite the processed Bloom with the final image .
I just know the theory for now..... a pointer to actual implementation would help a lot .  :)
Thanks in advance. :)
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on April 30, 2017, 10:30:58 am
Just use some additional render targets?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on April 30, 2017, 11:57:12 am
   I tried something like this->
Code: [Select]
                        fb.setRenderTarget(target1);

fb.setBlittingShader(shader1);
fb.clear(new RGBColor(123, 223, 237));
world.renderScene(fb);
world.draw(fb);
fb.display();


fb.setRenderTarget(target2);
fb.setBlittingShader(shader2);
fb.clear(new RGBColor(123, 223, 237));
world.renderScene(fb);
world.draw(fb);
fb.display();
fb.removeRenderTarget();

fb.blit(target2, 0, 0, 0, fb.getHeight(), target1.getWidth(), target1.getHeight() , fb.getWidth(), -fb.getHeight(), -1, false);



But this only shows me the output of second pass , ignoring the first one .
Am i missing something or do I have to remove something ?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 01, 2017, 05:40:25 pm
Where the relation between target1 and target2 in that code? You render into target1, then into target2 and then you blit target2. I don't see where target1 is actually used here!?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 01, 2017, 06:04:28 pm
Yes ... this code is not correct :(
there is something missing ...
But that's what my question is .... How to set this relation between target1 and target2 .
What is happening actually is , the target1 is having the vertical blurring . Now  this target1 texture has to be fed to the input of second pass....
So that the already vertically blurred image can be processed again with horizontal blur . You mentioned that this can be achieved with 2 render targets .So for now  I have created 2 render Targets... but then I was blank....I am not sure how to do this part in jpct ae.
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 02, 2017, 08:25:32 am
How is the general idea? You render the scene into target1, which includes the first blur. Then you want to use the same texture (target1) as the source for the other blur step and blit the resulting image?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 02, 2017, 08:41:13 am
Yes exactly!
At least this is what should happen according to my understanding... in theory,
So Yes I need to know jpct ae steps to execute the same idea .
Is this realizable this way ?
 Or....
If you have some alternate feasible / better way, please suggest. :)

Cause I would be then using this idea further for Post processing blur , So i guess I would have to do a third pass as well, for combining the blurred bright texture with the normal rendered image texture...
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 02, 2017, 09:13:50 am
I would try something like this:


In your code, you render the scene twice and I'm not sure if that's needed here. If target2's size isn't the same size as the screen, you might have to adjust the blitting coordinates. Here's some basic example that shows this: http://www.jpct.net/forum2/index.php/topic,4609.msg31858.html#msg31858 (http://www.jpct.net/forum2/index.php/topic,4609.msg31858.html#msg31858)

Another approach is to use an Overlay instead of blit(). You can obtain the Object3D from an overlay and assign your own shader to that one (just like you would with setBlittingShader()). That method has to no need to convert the coordinates.
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 02, 2017, 08:51:57 pm
Hi Egon,

You said
Quote
blit target1 fullscreen into target2
.
I got confused with this...did you mean that I should simply use fb.blit() ?? just after setting rendering to target 2 with blurring ?
like this->
Code: [Select]
// the first step you mentioned
                        fb.setRenderTarget(target1);
fb.setBlittingShader(shader1);
fb.clear(new RGBColor(123, 223, 237));
world.renderScene(fb);
world.draw(fb);
fb.display();
//the 2nd step->
fb.setRenderTarget(target2);
fb.setBlittingShader(shader2);
fb.blit(target1, 0, 0, 0, fb.getHeight(), target1.getWidth(), target1.getHeight() , fb.getWidth(), -fb.getHeight(), -1, false);   // this ? I'm not sure...

// the 3rd step->
fb.removeRenderTarget();
fb.blit(target2, 0, 0, 0, fb.getHeight(), target1.getWidth(), target1.getHeight() , fb.getWidth(), -fb.getHeight(), -1, false);


this did not worked... so this must be wrong, I know ...
But I'm asking this... because I failed to find any method which allows me to blit a texture to another in fullscreen...
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: AeroShark333 on May 03, 2017, 01:14:00 am
Is using an shader for an Overlay-Object3D here better than using a blitshader?
I think effectively you can reach the same goals..? But what would be faster here?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 03, 2017, 08:59:45 am
You said
Quote
blit target1 fullscreen into target2
.
I got confused with this...did you mean that I should simply use fb.blit() ?? just after setting rendering to target 2 with blurring ?
Yes, that's what I meant. In which way does it not work?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 03, 2017, 09:01:06 am
Is using an shader for an Overlay-Object3D here better than using a blitshader?
I think effectively you can reach the same goals..? But what would be faster here?
It shouldn't really matter. Which one is better is merely a matter of taste. Blitting into render targets can require some coordinate adjustments, so using an Overlay might be more intuitive for some though.
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: AeroShark333 on May 03, 2017, 01:06:29 pm
Is using an shader for an Overlay-Object3D here better than using a blitshader?
I think effectively you can reach the same goals..? But what would be faster here?
It shouldn't really matter. Which one is better is merely a matter of taste. Blitting into render targets can require some coordinate adjustments, so using an Overlay might be more intuitive for some though.
And performance-wise?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 03, 2017, 01:18:06 pm
It depends. I would say it's like the difference between starting a marathon with your left or with your right leg...
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 03, 2017, 09:04:57 pm
Quote
In which way does it not work?
It displays some weird artifacts like this (please see attachment)

and after few seconds , it crashes due to out of memory exception... :(

The Code is the same , the earlier posted code....
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 03, 2017, 09:16:17 pm
That looks quite wrong. Can you create a test case for this that I can compile and see for myself?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 03, 2017, 10:05:47 pm
Yes, Sure...
Below is my onDrawFrame()  ->
Code: [Select]
public void onDrawFrame(GL10 gl) {

if (this.hasToCreateBuffer) {
Logger.log("Recreating buffer...");
hasToCreateBuffer = false;
fb = new FrameBuffer(w, h);

}

if (touchTurn != 0) {

plane.rotateY(touchTurn);
plane1.rotateY(touchTurn);

touchTurn = 0;
}

if (touchTurnUp != 0) {
plane.rotateX(touchTurnUp);

plane1.rotateX(touchTurnUp);

touchTurnUp = 0;
}




fb.setRenderTarget(target1);
fb.setBlittingShader(shader1);
fb.clear(new RGBColor(123, 223, 237));
world.renderScene(fb);
world.draw(fb);
fb.display();

fb.setRenderTarget(target2);
fb.setBlittingShader(shader2);
fb.blit(target1, 0, 0, 0, fb.getHeight(), target1.getWidth(), target1.getHeight() , fb.getWidth(), -fb.getHeight(), -1, false);


fb.removeRenderTarget();

fb.blit(target2, 0, 0, 0, fb.getHeight(), target1.getWidth(), target1.getHeight() , fb.getWidth(), -fb.getHeight(), -1, false);

blitNumber(lfps, 5, 5);
            Log.e("fps ","f p s = " +lfps );
//fb.display();

if (System.currentTimeMillis() - time >= 1000) {
lfps = fps;
fps = 0;
time = System.currentTimeMillis();
}
fps++;
}



Is this enough ?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 04, 2017, 08:31:10 am
Can't you just zip your project and send it to me? That would make it much easier for me to get something similar up and running.
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 04, 2017, 09:51:25 am
Oh, and apart from that...your code misses the display()-calls required. That would explain the OOM errors as well, because it creates buffer after buffer without actually using them.
After blitting, you have to add a call to display() to actually swap the display buffers/render into the render target.

See the example that I put a link to above for...well, an example...
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 04, 2017, 10:07:40 am
Here's an example for desktop jPCT:

Code: [Select]
import java.awt.Color;

import com.threed.jpct.FrameBuffer;
import com.threed.jpct.GLSLShader;
import com.threed.jpct.IRenderer;
import com.threed.jpct.Loader;
import com.threed.jpct.Texture;
import com.threed.jpct.World;


public class RTT
{
  private World world;
  private FrameBuffer buffer;


  public static void main(String[] args)
    throws Exception
  {
    new RTT().loop();
  }


  private void loop()
    throws Exception
  {
    buffer = new FrameBuffer(640, 480, FrameBuffer.SAMPLINGMODE_NORMAL);
    buffer.disableRenderer(IRenderer.RENDERER_SOFTWARE);
    buffer.enableRenderer(IRenderer.RENDERER_OPENGL);

    world = new World();

    Texture t = new Texture("d.png");
    Texture t1 = new Texture(512, 512);
    Texture t2 = new Texture(512, 512);

    GLSLShader shady = new GLSLShader(Loader.loadTextFile("vertex.glsl"), Loader.loadTextFile("fragment.glsl"));
    shady.setUniform("colorMap", 0);

    while (!org.lwjgl.opengl.Display.isCloseRequested())
    {
      buffer.setRenderTarget(t1);
      buffer.setVirtualDimensions(buffer.getWidth(), buffer.getHeight());

      buffer.clear(Color.BLUE);
      world.renderScene(buffer);
      world.draw(buffer);

      buffer.blit(t, 0, 0, 0, 0, t.getWidth(), t.getHeight(), buffer.getWidth(), buffer.getHeight(), -1, false);

      display();
      buffer.removeRenderTarget();
     
      buffer.setRenderTarget(t2);
      buffer.setBlittingShader(shady);
      buffer.blit(t1, 0, 0, 0, 0, t1.getWidth(), t1.getHeight(), buffer.getWidth(), buffer.getHeight(), -1, false);
     
      display();
      buffer.removeRenderTarget();
      buffer.setBlittingShader(null);

      buffer.blit(t2, 0, 0, 0, 0, t1.getWidth(), t1.getHeight(), buffer.getWidth(), buffer.getHeight(), -1, false);
      display();

      Thread.sleep(10);
    }
    buffer.disableRenderer(IRenderer.RENDERER_OPENGL);
    buffer.dispose();
    System.exit(0);
  }


  public void display()
  {
    buffer.update();
    buffer.displayGLOnly();
  }
}


Vertex shader
Code: [Select]
varying vec2 texCoord;

void main(void)
{
gl_Position = ftransform();
texCoord = gl_MultiTexCoord0.xy;
}

Fragment shader
Code: [Select]
varying vec2 texCoord;

uniform sampler2D colorMap;

void main (void)
{
vec4 base = texture2D(colorMap, texCoord);
gl_FragColor = vec4(base.r*2.0,0.0,base.g,base.a);
}

What this basically does, is to blit a texture into a render target, set a blitting shader that eliminates green and boosts red and blit the first target texture into a second one using that shader. It then displays the result by blitting the second target onto the screen.
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 04, 2017, 12:47:48 pm
Ohh ! I missed the display method ! I see . . .

I'll check the code snippet you provided as soon I reach home... and let you know ... :)

and yes ... sorry for the inconvenience .
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 05, 2017, 09:15:26 pm
Hello Egon,
I tried to port your Desktop snippet into my demo...But for now it displays me red screen(My Default NPOT texture color)
I'm Uploading a  test Case zip...
And I have few more questions->
1]]
My logcat view was flooded with this->
Code: [Select]
05-05 23:16:09.875 27957-27975/? D/jPCT-AE: Binding buffers (1/1)!
05-05 23:16:09.876 27957-27975/? D/jPCT-AE: Binding texture 2
05-05 23:16:09.876 27957-27975/? D/jPCT-AE: Binding texture 3
05-05 23:16:09.877 27957-27975/? D/jPCT-AE: Binding texture 4
05-05 23:16:09.877 27957-27975/? D/jPCT-AE: Binding texture 5
05-05 23:16:09.877 27957-27975/? D/jPCT-AE: Buffer switches: 4
05-05 23:16:09.877 27957-27975/? D/jPCT-AE: Unbinding buffers (1)!
So is this Normal ? Because It buries other debug messages ...


2]]
if I remove blitnumber() method ...(or any fb.blit() in general) which is present immediately after my fb.blit(target....) then it ignores my shader applied on to the screen and displays me normal render .
Is this behavior normal ?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 06, 2017, 07:10:00 pm
I downloaded your example, but I'm not sure what to make of it. You have a shader1, that you never use. You have a target and a target2 that you never use...!?

I'm not sure how it's supposed to work, but the basic idea would be something like this:


Also keep in mind that assigning a blitting shader doesn't do anything if you don't blit anything. In your code, you don't assign shader1 at all, then you set "target" as a render target, render the scene in "target", blit "target1" (which is red) all over it and then blit the (still red) "target1" into the actual frame buffer without a target. I fail to see the point in that.

About the second question...that might be related to the blitting buffers. In case of doubt, render the dummy world after blitting if all you do in one pass is blitting . That will flush the blitting buffer for sure. Or, as a cheap alternative, blit a transparent texture onto the final image (1*1 pixel is enough).
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 06, 2017, 07:25:54 pm
Sorry My bad.... I had actually uploaded the wrong file ! :-\
I have re-uploaded the file ...
which is giving me red screen...
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 07, 2017, 06:53:34 pm
Hmm...that one refers to grass, face_norm and img...neither of which is in the package... ???
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 07, 2017, 07:27:37 pm
Ohh yes! ,I removed them because it was exceeding the file size limit....they were quite big ~512*512 ...
May I please ask you to find this attached image and replace the grass and face_norm with the same ?

Sorry for the inconvenience caused , I am feeling embarrassed now :-[
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 07, 2017, 08:53:21 pm
Ok, I had a look, but I wasn't able to compile the shaders (tried on PowerVR). You are using some 3.0 language constructs, so that I would have to set that version but if you do, a lot of stuff has to be changed as well like (varying -> out, attribute -> in, texture2D() -> texture()) and while I tried to do that, I ran into some issue that I didn't have the time solve ATM.

Anyway, I still think that your render logic is flawed. I think it should look something like this:

Code: [Select]
fb.setVirtualDimensions(target.getWidth(), target.getHeight());

// Render scene into target
fb.setRenderTarget(target);
fb.clear(new RGBColor(123, 223, 237));
world.renderScene(fb);
world.draw(fb);
fb.display();
fb.removeRenderTarget();

// Blur target by blitting it into target1
fb.setRenderTarget(target1);
fb.setBlittingShader(shader1);
fb.blit(target, 0, 0, 0, fb.getHeight(), target1.getWidth(), target1.getHeight() , fb.getWidth(), -fb.getHeight(), -1, false);
                        // Maybe a transparent dummy blit here to flush the pipeline
fb.display();
fb.removeRenderTarget();

// Blur target1 by blitting it into target2
fb.setRenderTarget(target2);
fb.setBlittingShader(shader2);
fb.blit(target1, 0, 0, 0, fb.getHeight(), target1.getWidth(), target1.getHeight() , fb.getWidth(), -fb.getHeight(), -1, false);
                         // Maybe a transparent dummy blit here to flush the pipeline
fb.display();
fb.removeRenderTarget();
fb.setBlittingShader(null);

fb.blit(target2, 0, 0, 0, fb.getHeight(), target2.getWidth(), target2.getHeight() , fb.getWidth(), -fb.getHeight(), -1, false);
fb.display();
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 08, 2017, 10:19:15 am
I'll try this code soon... :)
But regarding the shader problem , it compiles fine on my pretty old device(snapdragon 410 , with adreno 306 , 2 GB RAM ).
I'll try to get my hands on the PowerVR device for verifying, if possible . BTW , which device was used for testing ?
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 08, 2017, 10:58:00 am
A Redmi Note 2, which was the cheapest one that I could get with a PowerVR 6xxx chip.
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: SonicBurst2 on May 11, 2017, 09:25:22 pm
Thank you Egon !...
The Code snippet worked.. :)
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on May 12, 2017, 08:17:47 am
Cool!
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: AeroShark333 on June 19, 2017, 01:22:30 am
I was just wondering about this multipass rendering thing...
Is it not possible to use one single rendertarget for multipass rendering? (so with every pass the rendertarget is used for blitting)
I wonder why in this thread at least 2 rendertarget textures were needed.

In this thread:
Render world into rendertarget1.
Blit rendertarget1 into rendertarget2 (with blit shader 1)
Blit rendertarget2 into the normal framebuffer (with blit shader 2)

But why not just this?:
Render world into rendertarget1.
Blit rendertarget1 into rendertarget1 (with blit shader 1)
Blit rendertarget1 into the normal framebuffer (with blit shader 2)

I kind of tried the second method but it didn't really work...
Though I wonder why it didn't work
Title: Re: Gaussian Blur |Post Processing Bloom| MultiPass Rendering in jpct ae
Post by: EgonOlsen on June 19, 2017, 07:57:10 am
You can't use the an input buffer as an output buffer at the same time.