Hello. In my previous thread I figured out how to render directly to a Canvas using JAWT and EGL+GLES
http://forum.lwjgl.org/index.php?topic=6970.0

But now I have a different problem. The Application that I'm supposed to integrate this into is built using jLayeredFrame. However if I put my Canvas into it - the canvas is invisible on all layers except the topmost one. In other words, if anything else should be drawn on top of my Canvas - it completely disappears.

I tried putting it into jPanel as a container, and it exhiboits the same behavior. If jPanel is the topmost element - both it and Canvas can be drawn. If it's below something - only jPanel gets drawn, but not the canvas.

Anyway, if jPanel works, I thought I would render to that instead. However, I can't figure it out. Here's a bit of code:

private Canvas canvas;
    private JPanel panel;
    private JAWTDrawingSurface ds;
    public long display;
    private long eglDisplay;
    public long drawable;
    private long surface;

    public static final JAWT awt;
    static {
        awt = JAWT.calloc();
        awt.version(JAWT_VERSION_1_4);
        if (!JAWT_GetAWT(awt))
            throw new AssertionError("GetAWT failed");
    }

public void Init2()
    {
        int error;

        System.out.println("Window init2() started");

        this.ds = JAWT_GetDrawingSurface(panel, awt.GetDrawingSurface());
        //JAWTDrawingSurface ds = JAWT_GetDrawingSurface(canvas, awt.GetDrawingSurface());
        try
        {
            lock();
            try
            {
                JAWTDrawingSurfaceInfo dsi = JAWT_DrawingSurface_GetDrawingSurfaceInfo(ds, ds.GetDrawingSurfaceInfo());

                JAWTX11DrawingSurfaceInfo dsiWin = JAWTX11DrawingSurfaceInfo.create(dsi.platformInfo());

                int depth = dsiWin.depth();
                this.display = dsiWin.display();
                this.drawable = dsiWin.drawable();
                long x = dsiWin.colormapID();
                long y = dsiWin.visualID();

                System.out.printf("EGL Display %d, drawable: \n", display, drawable);

                eglDisplay = eglGetDisplay(display);

                error = eglGetError();

                EGLCapabilities egl;
                try (MemoryStack stack = stackPush()) {
                    IntBuffer major = stack.mallocInt(1);
                    IntBuffer minor = stack.mallocInt(1);

                    if (!eglInitialize(eglDisplay, major, minor)) {
                        throw new IllegalStateException(String.format("Failed to initialize EGL [0x%X]", eglGetError()));
                    }

                    egl = EGL.createDisplayCapabilities(eglDisplay, major.get(0), minor.get(0));
                }
                System.out.println("EGL caps created");

                error = eglGetError();

                IntBuffer attrib_list = BufferUtils.createIntBuffer(18);
                attrib_list.put(EGL_CONFORMANT).put(EGL_OPENGL_ES2_BIT);
                attrib_list.put(EGL_DEPTH_SIZE).put(24);
                attrib_list.put(EGL_NONE);
                attrib_list.flip();

                PointerBuffer fbConfigs = BufferUtils.createPointerBuffer(1);
                IntBuffer numConfigs = BufferUtils.createIntBuffer(1);

                boolean test2 = eglChooseConfig(eglDisplay, attrib_list, fbConfigs,numConfigs);

                if (fbConfigs == null || fbConfigs.capacity() == 0) {
                    // No framebuffer configurations supported!
                    System.out.println("No supported framebuffer configurations found");
                }

                long test = numConfigs.get(0);

                IntBuffer context_attrib_list = BufferUtils.createIntBuffer(18);
                context_attrib_list.put(EGL_CONTEXT_MAJOR_VERSION).put(3);
                context_attrib_list.put(EGL_CONTEXT_MINOR_VERSION).put(0);
                context_attrib_list.put(EGL_NONE);
                context_attrib_list.flip();

                long context = eglCreateContext(eglDisplay,fbConfigs.get(0),EGL_NO_CONTEXT,context_attrib_list);

                error = eglGetError();

                surface = eglCreateWindowSurface(eglDisplay,fbConfigs.get(0),drawable,(int[])null);

                error = eglGetError();

                eglMakeCurrent(eglDisplay,surface,surface,context);

                error = eglGetError();

                GLESCapabilities gles = GLES.createCapabilities();
                System.out.println("CLES caps created");

                JAWT_DrawingSurface_FreeDrawingSurfaceInfo(dsi, ds.FreeDrawingSurfaceInfo());
            }
            finally
            {
                unlock();

            }

        }
        catch (Exception e)
        {
            System.out.println("JAWT Failed");
        }
if I pass Canvas to JAWT_GetDrawingSurface() - everything works fine.
But if I pass the jPanel - then it SEEMS to work fine, except not really. It doesn't throw any exceptions, and the display seems to be all good - both EGL and GLES are initialized just fine through it. The FrameBuffer objects are also correctly found.

But when it comes to eglCreateWindowSurface() - it generated EGL_BAD_ALLOC error code.
Also, you can see that something is wrong on the int depth = dsiWin.depth(); line. Because this parameter is supposed to be depth buffer size in bits for rendering context - and with Canvas it equals to 24. With jPanel it equals 32638 - which is OBVIOUSLY wrong. I suspect that while the display info is correct - everything else isn't, including drawable address, and thus you get EGL_BAD_ALLOC.

Which is a bit strange, because JAWT_GetDrawingSurface() requires awt.Component as input, which both Canvas and jPanel inherit from.

But anyway, can someone help me figure it out?

The problem is how Swing is written on top of AWT. There are so called "heavyweight" and "lightweight" UI components.

Heavyweight means: The component actually has a 1:1 relation to the underlying OS's window manager and its UI component.
AWT calls the window-manager equivalents its "peer".

Lightweight is what most Swing components are, which are not managed by the underlying OS's window manager but by AWT. So for all lightweight Swing components, all window events (such as paint requests and mouse/keyboard events) are intercepted by the heavyweight AWT component the lightweight component is added to (such as a Frame or a Canvas) and processed solely by Swing.
That means lightweight Swing components also do not have a window handle (hWnd on Windows) because they are no UI components from the OS's point of view. That's why you also cannot create an OpenGL context for them, because the OS does not have any handle to lightweight Swing components.

So, if we also look at the documentation of the AWT Native Interface: https://docs.oracle.com/javase/10/docs/specs/AWT_Native_Interface.html
and search for the function prototype of "GetDrawingSurface" it says in its documentation

Return a drawing surface from a target jobject.  This value
may be cached.
Returns NULL if an error has occurred.
Target must be a java.awt.Component (should be a Canvas
or Window for native rendering).
FreeDrawingSurface() must be called when finished with the
returned JAWT_DrawingSurface.

So, not every java.awt.Component is possible as an argument here. Only Window and Canvas, which are heavyweight components, so they actually have a native peer with a window handle the OS can use.

Ok...
So, what do I do? I still have the jLayeredPane that handles all of the UI in the main app.
How can I attach an AWT Canvas to it in a way that it gets properly drawn on underlying layers.

OR

How can the interface be re-written to allow it? Maybe there is some kind of alternative to jLayeredPane approach?

in Swing, jFrame seems to be a heavyweight component. My routine seems to generate valid code and surfaces and whatnot.
But somehow I can't render directly to jFrame as I can to a Canvas. Why?

In the end, I had zero problem with Z-layering. What I had problems with is elements - even invisible ones - cutting out shapes in Canvas.

The thing is, the way Swing and AWT interaction is done is thus:
Heavyweight elements DO display on top of lightweights ALWAYS. It may not seem like that, but they really do, because all of the displaying of lightweights is done essentially by the closest heavyweight Ancestor. Thus, if Canvas belongs to a JPanel, and lightweights also belong to JPanel - then you essentially have two "Windows" drawn - JPanel and it's child window Canvas on top of it.

The way java implemented a workaround is a bit convoluted. Basically, any element that is Z-ordered above canvas - cuts a hole in your canvas and peeks through it. The illusion is that your lightweight is on top of Canvas - but it really is below. It simply cut a hole.
The problem with that is that you get problems like this:
https://stackoverflow.com/questions/59321794/swing-layering-transparent-component-ignores-underlying-awt-element

I've spent quite a bit of time trying to figure out exactly what's going on and how to work around that. And as you can see in the above link - there's a solution to make your elements at least of a desired shape, or hide them completely if they're just a container that's supposed to be transparent anyway.
The downside that you can't really make complex elements transparent. Like an icon of a complex shape - the best you can do is encase it in a circle but it will be an opaque circle.

If you absolutely MUST make something transparent - there is a workaround you can implement. Namely, you can display your transparent elements in the second transparent jFrame that sits on top of the first one. Set JFrame type to Undecorated, make all elements on it (except the one you need to show Opaque=false, set the JFrame's background to Color(1,1,1,0), where 0 is alpha. I.e. transparent. This is not at all an elegant solution, but hey - an option is still better than nothing.