Opengl Gl_Polygon Concave Polygon Doesn't Color In

OpenGL GL_POLYGON concave polygon doesn't color in

GL_POLYGON is only for convex polygons:

GL_POLYGON: Draws a single, convex polygon. Vertices 1 through N define this polygon.

For concave polygons you have at least two options:

1. Triangulate the polygon and use GL_TRIANGLES.

2. Use the stencil buffer trick.

OpenGL GL_POLYGON not printing correctly

The behavior of GL_POLYGON is only defined for convex polygons. Your polygon is concave, so the behavior of this function is undefined (and, as you've observed, it doesn't always work).

You will need to break your polygon up into one or more convex polygons for this to work correctly. In your case, one easy way of doing this would be to "cut" horizontally across the polygon:

(50,60) _____(70,60)
       |    /
       |   /
(50,40)|__/(60,40)
       |  \
       |   \
(50,20)|____\(70,20)

Use of GL_STENCIL_TEST to render concave polygons

...
glClearStencil(0);
...

Be aware that glClearStencil() just sets a bit of state and doesn't actually clear the stencil buffer.

Try adding a glClear( GL_STENCIL_BUFFER_BIT ) somewhere before each polygon.

EDIT: Like this:

#include <GL/glut.h>
#include <glm/glm.hpp>
#include <glm/gtc/type_ptr.hpp>
#include <vector>

std::vector< glm::vec2 > pts;
bool leftHeld = true;
glm::vec2* dragPt = NULL;
void mouse( int button, int state, int x, int y )
{
    glm::vec2 pt( x, glutGet( GLUT_WINDOW_HEIGHT ) - y );

    // left mouse button starts dragging a point
    dragPt = NULL;
    leftHeld = false;
    if( button == GLUT_LEFT_BUTTON && state == GLUT_DOWN )
    {
        leftHeld = true;
        size_t minIdx = 0;
        for( size_t i = 0; i < pts.size(); ++i )
        {
            float newDist = glm::distance( pt, pts[ i ] );
            float oldDist = glm::distance( pt, pts[ minIdx ] );
            if( newDist <= oldDist && newDist < 15.0f )
            {
                minIdx = i;
                dragPt = &pts[ minIdx ];
            }
        }
    }

    // middle mouse button clears all points
    if( button == GLUT_MIDDLE_BUTTON && state == GLUT_UP )
    {
        pts.clear();
    }

    // right mouse button adds a point
    if( button == GLUT_RIGHT_BUTTON && state == GLUT_UP )
    {
        pts.push_back( pt );
    }

    glutPostRedisplay();
}

void motion( int x, int y )
{
    glm::vec2 pt( x, glutGet( GLUT_WINDOW_HEIGHT ) - y );
    if( dragPt && leftHeld )
    {
        *dragPt = pt;
        glutPostRedisplay();
    }
}

void glLine( const std::vector< glm::vec2 >& line, GLenum mode )
{
    glBegin( mode );
    for( size_t i = 0; i < line.size(); ++i )
    {
        glVertex2f( line[i].x, line[i].y );
    }
    glEnd();
}

void display()
{
    glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );

    glMatrixMode( GL_PROJECTION );
    glLoadIdentity();
    double w = glutGet( GLUT_WINDOW_WIDTH );
    double h = glutGet( GLUT_WINDOW_HEIGHT );
    glOrtho( 0, w, 0, h, -1, 1 );

    glMatrixMode( GL_MODELVIEW );
    glLoadIdentity();

    // draw polygon
    glClear( GL_STENCIL_BUFFER_BIT );
    {
        // fill stencil buffer
        glEnable( GL_STENCIL_TEST );
        glColorMask( GL_FALSE,GL_FALSE,GL_FALSE,GL_FALSE );
        glStencilOp( GL_KEEP, GL_KEEP, GL_INVERT );
        glStencilFunc( GL_ALWAYS, 0x1, 0x1 );
        glBegin( GL_TRIANGLES );
        for( size_t i = 1; i+1 < pts.size(); ++i )
        {
            glVertex2fv( glm::value_ptr( pts[ 0 ] ) );
            glVertex2fv( glm::value_ptr( pts[ i ] ) );
            glVertex2fv( glm::value_ptr( pts[ i+1 ] ) );
        }
        glEnd();

        // fill color buffer
        glColor3ub( 0, 128, 0 );
        glColorMask( GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE );
        glStencilFunc( GL_EQUAL, 0x1, 0x1 );
        glBegin( GL_TRIANGLES );
        for( size_t i = 1; i+1 < pts.size(); ++i )
        {
            glVertex2fv( glm::value_ptr( pts[ 0 ] ) );
            glVertex2fv( glm::value_ptr( pts[ i ] ) );
            glVertex2fv( glm::value_ptr( pts[ i+1 ] ) );
        }
        glEnd();
        glDisable( GL_STENCIL_TEST );
    }

    // draw polygon boundary
    glLineWidth( 1 );
    glColor3ub( 255, 255, 255 );
    glLine( pts, GL_LINE_LOOP );

    // draw vertexes
    glPointSize( 9 );
    glColor3ub( 255, 0, 0 );
    glLine( pts, GL_POINTS );

    glutSwapBuffers();
}

int main( int argc, char **argv )
{
    glutInit( &argc, argv );
    glutInitDisplayMode( GLUT_RGBA | GLUT_DEPTH | GLUT_DOUBLE | GLUT_STENCIL );
    glutInitWindowSize( 640, 480 );
    glutCreateWindow( "GLUT" );
    glutMouseFunc( mouse );
    glutMotionFunc( motion );
    glutDisplayFunc( display );
    glutMainLoop();
    return 0;
}

Vertex not read in opengl

This is your polygon:

Your shape is not convex and GL_POLYGON stipulates:

GL_POLYGON

Draws a single, convex polygon. Vertices 1 through N define this polygon.

To rasterize this contour, manually break your shape into triangles, or other primitive components such as a TRIANGLE FAN or TRIANGLE STRIP. This is called tessellation. Although GL_POLYGON looks tempting, it is for convex shapes only.

(original image source)

For arbitrary polygons, you'll need to tessellate the contour yourself. You can seek out a library for doing this such as: https://github.com/mapbox/earcut.hpp The good news is that you only have to do this once, and you can do it offline (not at run-time). Once your polygon is tessellated, you'll be able draw it with GL_TRIANGLES.

OpenGL GL_POLYGON concave polygon doesn't color in

GL_POLYGON is only for convex polygons:

GL_POLYGON: Draws a single, convex polygon. Vertices 1 through N define this polygon.

For concave polygons you have at least two options:

1. Triangulate the polygon and use GL_TRIANGLES.

2. Use the stencil buffer trick.

How to force openGL to draw a non-convex filled polygon

Note: This answer shares content with my recent answer to a similar question here: Black out everything outside a polygon. I did not nominate the questions as duplicates because they sound different, and could potentially have different answers, even though this one happens to be mostly the same.

One approach for drawing non-convex polygons is to break them down into triangles. There are a number of algorithms that can do this, which can be found by searching for keywords like "polygon triangulation".

OpenGL has another mechanism that works great for this: stencil buffers. An explanation of this approach is in the Red Book under Drawing Filled, Concave Polygons Using the Stencil Buffer. The main idea is that one can draw a triangle fan with an arbitrary origin and your polygon vertices. The pixels that are inside the polygon will then be drawn an odd number of times, while the pixels outside the polygons are drawn an even number of times. The stencil buffer is used to track the odd/even count.

To outline the main steps:

1. While setting up the context and drawing surface, make sure that a configuration with a stencil buffer is requested.

2. During drawing, clear the stencil buffer along with the color buffer, and enable the stencil test.

   glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
   glEnable(GL_STENCIL_TEST);
   

3. Set up state for the render pass that counts if pixels are rendered an odd/even number of times. Note that this must only write to the stencil buffer, so color writes are disabled. The key part is the GL_INVERT for the stencil op, which flips the stencil value each time a pixel is rendered, which ends up storing the odd/even count in the stencil buffer.

   glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
   glStencilFunc(GL_ALWAYS, 0, 1);
   glStencilOp(GL_KEEP, GL_KEEP, GL_INVERT);
   glStencilMask(1);
   

4. Render a triangle fan with an arbitrary point, e.g. (0.0, 0.0), as the first vertex, and the polygon corners as the remaining vertices. The polygon must be closed, so the first and last polygon corner must be the same. If p1, p2, ... , pN are the polygon corners, the sequence of vertices for the GL_TRIANGLE_FAN draw call is:

   (0.0f, 0.0f), p1, p2, ... , pN, p1
   

   A trivial shader can be used for this pass since the color value is not even written.

5. Enable color writes again, and set up the stencil test attributes to render only pixels that were rendered an odd number of times in the previous pass.

   glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
   glStencilFunc(GL_EQUAL, 1, 1);
   glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
   

6. Draw geometry that covers the entire area of the polygon, and possibly more. This can for example be the triangle fan from step 4, or a bounding box of the polygon. Only the part within the polygon outline will be rendered, the rest is eliminated by the stencil test.

Getting errors when trying to draw complex polygons with triangles in OpenGL

You can think of the problem in two stages consisting of turning the polygon into convex sub-polygons, and then triangulate each of the sub-polygons. The algorithm to triangulate a sub polygon (triangulatePoly) is a fairly simple recursive function that takes in a polygon and checks if it has 3 points. If it does, it returns, if not, it creates a triangle from the first 3 points adds it to a list and decrements the polygon by that triangle, leaving you with a list of triangles that comprise the polygon.

The convex sub-polygon algorithm (decomposePoly) is harder to explain as it is quite complicated and so if you want to understand it, it is here.

Finally, here is an implementation, written with OpenGL2 and quite clustered for brevity.

// ######################

public class Point {
public float x;
public float y;

public Point(float _x, float _y) {
    x = _x;
    y = _y;
}

public static float area(Point a, Point b, Point c) {
    return (((b.x - a.x)*(c.y - a.y))-((c.x - a.x)*(b.y - a.y)));
}

public static boolean left(Point a, Point b, Point c) {
    return area(a, b, c) > 0;
}

public static boolean leftOn(Point a, Point b, Point c) {
    return area(a, b, c) >= 0;
}

public static boolean rightOn(Point a, Point b, Point c) {
    return area(a, b, c) <= 0;
}

public static boolean right(Point a, Point b, Point c) {
    return area(a, b, c) < 0;
}

public static float sqdist(Point a, Point b) {
    float dx = b.x - a.x;
    float dy = b.y - a.y;
    return dx * dx + dy * dy;
}
} 

// ######################

import java.util.Vector;
public class Polygon extends Vector<Point> {
        @Override
        public Point get(int i)  {
            // hacky way of getting the modulo
            return super.get(((i % this.size()) + this.size()) % this.size());
        }
}

// ######################

import org.lwjgl.*;
import org.lwjgl.glfw.*;
import org.lwjgl.opengl.*;
import org.lwjgl.system.*;

import java.nio.*;

import static org.lwjgl.glfw.Callbacks.*;
import static org.lwjgl.glfw.GLFW.*;
import static org.lwjgl.opengl.GL11.*;
import static org.lwjgl.system.MemoryStack.*;
import static org.lwjgl.system.MemoryUtil.*;

import java.util.Collections;
import java.util.Vector;

public class DecomposePolyExample {

    private long window;

    private int WIDTH = 300;
    private int HEIGHT = 300;

    private float mouse_x = WIDTH / 2;
    private float mouse_y = HEIGHT / 2;

    private Polygon incPoly = new Polygon();

    private Vector<Polygon> polys = new Vector<Polygon>();
    private Vector<Polygon> tris = new Vector<Polygon>();
    private Vector<Point> steinerPoints = new Vector<Point>();
    private Vector<Point> reflexVertices = new Vector<Point>();

    private boolean polyComplete = false;

    public void run() {
        System.out.println("Hello LWJGL" + Version.getVersion() + "!");

        init();
        loop();

        // Free the window callbacks and destroy the window
        glfwFreeCallbacks(window);
        glfwDestroyWindow(window);

        // Terminate GLFW and free the error callback
        glfwTerminate();
        glfwSetErrorCallback(null).free();
    }

    private void init() {

        // Setup and error callback. The default implementation
        // will print the error message in System.err.
        GLFWErrorCallback.createPrint(System.err).set();

        // Initialize GLFW. Most GLFW functions will not work before doing this.
        if (!glfwInit()) {
            throw new IllegalStateException("Unable to initialize GLFW");
        }

        // Create the window
        window = glfwCreateWindow(WIDTH, HEIGHT, "Hello World!", NULL, NULL);
        if (window == NULL) {
            throw new RuntimeException("Failed to create the GLFW window");
        }

        // Setup a key callback. It will be called every time a key is pressed, repeated or released.
        glfwSetKeyCallback(window, (window, key, scancode, action, mods) -> {
            if ( key == GLFW_KEY_ESCAPE && action == GLFW_RELEASE) {
                glfwSetWindowShouldClose(window, true); // We will detect this in the rendering loop
            }
        });

        glfwSetCursorPosCallback(window, (window, x, y) -> {
            mouse_x = (float)x;
            mouse_y = HEIGHT - (float)y;
        });

        glfwSetMouseButtonCallback(window, (window, button, action, mods) -> {
            if (action != GLFW_PRESS){
                return;
            }
            int lClick = glfwGetMouseButton(window, GLFW_MOUSE_BUTTON_LEFT);
            if (lClick == GLFW_PRESS)
            {
                Point p = new Point(mouse_x, mouse_y);
                incPoly.add(p);
            }
            int rClick = glfwGetMouseButton(window, GLFW_MOUSE_BUTTON_RIGHT);
            if (rClick == GLFW_PRESS)
            {
                polyComplete = true;
                incPoly = makeCCW(incPoly);
                decomposePoly(incPoly);
                triangulatePoly(polys); 
            }
        });

        // Make the OpenGL context current
        glfwMakeContextCurrent(window);
        // Enable v-sync
        glfwSwapInterval(1);

        // Make the window visible
        glfwShowWindow(window);
    }

    public Point toNDC(Point p) {
        float x = 2*p.x / WIDTH - 1;
        float y = 2*p.y / HEIGHT - 1;
        return new Point(x, y);
    }

    public Polygon makeCCW(Polygon poly) {
        int br = 0;

        // find bottom right point
        for (int i = 1; i < poly.size(); ++i) {
            if (poly.get(i).y < poly.get(br).y || (poly.get(i).y == poly.get(br).y && poly.get(i).x > poly.get(br).x)) {
                br = i;
            }
        }

        // reverse poly if clockwise
        if (!Point.left(poly.get(br - 1), poly.get(br), poly.get(br + 1))) {
            Collections.reverse(poly);
        }
       return poly;
    }

    public boolean isReflex(Polygon poly, int i) {  
        return Point.right(poly.get(i - 1), poly.get(i), poly.get(i + 1));
    }

    public boolean eq(float a, float b) {
        return Math.abs(a - b) <= 1e-8;
    }

    Point intersection(Point p1, Point p2, Point q1, Point q2) {
        Point i = new Point(0,0);
        float a1, b1, c1, a2, b2, c2, det;
        a1 = p2.y - p1.y;
        b1 = p1.x - p2.x;
        c1 = a1 * p1.x + b1 * p1.y;
        a2 = q2.y - q1.y;
        b2 = q1.x - q2.x;
        c2 = a2 * q1.x + b2 * q1.y;
        det = a1 * b2 - a2*b1;
        if (!eq(det, 0)) { // lines are not parallel
            i.x = (b2 * c1 - b1 * c2) / det;
            i.y = (a1 * c2 - a2 * c1) / det;
        }
        return i;
    }

    public void decomposePoly(Polygon poly) {
        Point upperInt = new Point(0,0);
        Point lowerInt = new Point(0,0);
        Point p = new Point(0,0);
        Point closestVert = new Point(0,0);

        float upperDist, lowerDist, d, closestDist;
        int upperIndex = 0;
        int lowerIndex = 0;
        int closestIndex = 0;
        Polygon lowerPoly = new Polygon();
        Polygon upperPoly = new Polygon();
        for (int i = 0; i < poly.size(); ++i) {
            if (isReflex(poly, i)) {
                reflexVertices.add(poly.get(i));
                upperDist = lowerDist = Float.MAX_VALUE;
                for (int j = 0; j < poly.size(); ++j) {
                    if (Point.left(poly.get(i - 1), poly.get(i), poly.get(j))
                            && Point.rightOn(poly.get(i - 1), poly.get(i), poly.get(j - 1))) { // if line intersects with an edge
                        p = intersection(poly.get(i - 1), poly.get(i), poly.get(j), poly.get(j - 1)); // find the point of intersection
                        if (Point.right(poly.get(i + 1), poly.get(i), p)) { // make sure it's inside the poly
                            d = Point.sqdist(poly.get(i), p);
                            if (d < lowerDist) { // keep only the closest intersection
                                lowerDist = d;
                                lowerInt = p;
                                lowerIndex = j;
                            }
                        }
                    }
                    if (Point.left(poly.get(i + 1), poly.get(i), poly.get(j + 1))
                            && Point.rightOn(poly.get(i + 1), poly.get(i), poly.get(j))) {
                        p = intersection(poly.get(i + 1), poly.get(i), poly.get(j), poly.get(j + 1));
                        if (Point.left(poly.get(i - 1), poly.get(i), p)) {
                            d = Point.sqdist(poly.get(i), p);
                            if (d < upperDist) {
                                upperDist = d;
                                upperInt = p;
                                upperIndex = j;
                            }
                        }
                    }
                }

                // if there are no vertices to connect to, choose a point in the middle
                if (lowerIndex == (upperIndex + 1) % poly.size()) {
                    p.x = (lowerInt.x + upperInt.x) / 2;
                    p.y = (lowerInt.y + upperInt.y) / 2;
                    steinerPoints.add(p);

                    if (i < upperIndex) {
                        for (int j = i; j < upperIndex + 1; j++) {
                            lowerPoly.add(poly.get(j));
                        }
                        lowerPoly.add(p);
                        upperPoly.add(p);
                        if (lowerIndex != 0) {
                            for (int j = lowerIndex; j < poly.size(); j++) {
                                upperPoly.add(poly.get(j));
                            }
                        }
                        for (int j = 0; j < i + 1; j++) {
                            upperPoly.add(poly.get(j));
                        }

                    } else {
                        if (i != 0) {
                            for (int j = 0; j < i; j++) {
                                lowerPoly.add(poly.get(j));
                            }
                        }
                        for (int j = 0; j < upperIndex + 1; j++) {
                            lowerPoly.add(poly.get(j));
                        }
                        lowerPoly.add(p);
                        upperPoly.add(p);
                        for (int j = lowerIndex; j < i + 1; j++) {
                            upperPoly.add(poly.get(j));
                        }
                    }
                } else {
                    // connect to the closest point within the triangle

                    if (lowerIndex > upperIndex) {
                        upperIndex += poly.size();
                    }
                    closestDist = Float.MAX_VALUE;
                    for (int j = lowerIndex; j <= upperIndex; ++j) {
                        if (Point.leftOn(poly.get(i - 1), poly.get(i), poly.get(j))
                                && Point.rightOn(poly.get(i + 1), poly.get(i), poly.get(j))) {
                            d = Point.sqdist(poly.get(i), poly.get(j));
                            if (d < closestDist) {
                                closestDist = d;
                                closestVert = poly.get(j);
                                closestIndex = j % poly.size();
                            }
                        }
                    }
                    if (i < closestIndex) {
                        for (int j = i; j < closestIndex + 1; j++) {
                            lowerPoly.add(poly.get(j));
                        }

                        if (closestIndex != 0) {
                            for (int j = closestIndex; j < poly.size(); j++) {
                                upperPoly.add(poly.get(j));
                            }
                        }
                        for (int j = 0; j < i + 1; j++) {
                            upperPoly.add(poly.get(j));
                        }
                    } else {
                        if (i != 0) {
                            for (int j = i; j < poly.size(); j++) {
                                lowerPoly.add(poly.get(j));
                            }
                        }
                        for (int j = 0; j < closestIndex + 1; j++) {
                            lowerPoly.add(poly.get(j));
                        }
                        for (int j = closestIndex; j < i + 1; j++) {
                            upperPoly.add(poly.get(j));
                        }
                    }
                }
                // solve smallest poly first
                if (lowerPoly.size() < upperPoly.size()) {
                    decomposePoly(lowerPoly);
                    decomposePoly(upperPoly);
                } else {
                    decomposePoly(upperPoly);
                    decomposePoly(lowerPoly);
                }
                return;
            }
        }
        polys.add(poly);
    }

    public void triangulatePoly(Vector<Polygon> polys) {
        for (int i = 0; i < polys.size(); i++) {
            Polygon poly = polys.get(i);
            // return if poly is a triangle
            if (poly.size() == 3) {
                tris.add(poly);
                polys.remove(i);
            }
            else {
                // split poly into new triangle and poly
                Polygon tri = new Polygon();
                for (int j = 0; j < 3; j++) {
                    tri.add(poly.get(j));
                }
                Polygon newPoly = new Polygon();
                newPoly.add(poly.get(0));
                for (int k = 2; k < poly.size(); k++) {
                    newPoly.add(poly.get(k));
                }
                polys.set(i, newPoly);
                tris.add(tri);
            }
        }
        if (polys.size() != 0) {
            triangulatePoly(polys);
        }
    }

    private void loop() {
        GL.createCapabilities();

        // Set the clear color
        glClearColor(0.0f, 0.0f, 0.0f, 0.0f);

        while (!glfwWindowShouldClose(window)) {
            glClear(GL_COLOR_BUFFER_BIT); // clear the framebuffer
            System.out.println(tris.size());
            if (!polyComplete) {
                GL11.glBegin(GL_LINE_STRIP);
                for (int i = 0; i < incPoly.size(); ++i) {
                    Point p_ndc = toNDC(incPoly.get(i));
                    GL11.glVertex2f(p_ndc.x, p_ndc.y);
                }
                GL11.glEnd();
            } else {
                // polygon outlines (thin)
                for (int i = 0; i < tris.size(); ++i) {
                    GL11.glBegin(GL_LINE_LOOP);
                    for (int j = 0; j < tris.get(i).size(); ++j) {
                        Point p_ndc = toNDC(tris.get(i).get(j));
                        GL11.glVertex2f(p_ndc.x, p_ndc.y);
                    }
                    GL11.glEnd();
                }

                GL11.glBegin(GL_LINE_LOOP);
                for (int i = 0; i < incPoly.size(); ++i) {
                    Point p_ndc = toNDC(incPoly.get(i));
                    GL11.glVertex2f(p_ndc.x, p_ndc.y);
                }
                GL11.glEnd();
            }

            glfwSwapBuffers(window); // swap the color buffers

            // Poll for window events. The key callback above will only be
            // invoked during this call.
            glfwPollEvents();
        }
    }

    public static void main(String[] args) {
        new DecomposePolyExample().run();
    }
}

Demo:

Why can't Pyglet draw a polygon correctly?

PyGlet is a OpenGL wrapper. Polygones which are drawn with the Legacy OpenGL Primitive type GL_POLYGON have to be Convex. Concave polygons might not be drawn correctly.

---

Use the primitive type GL_TRIANGLE_FAN and start with the point (59, 149). This will solve the issue in your special case:

point_list = [59, 149, 328, 204, 305, 284, 3, 197, 25, 107, 18, 61]
ec = int(len(point_list)/2)
batch.add(ec, pyglet.gl.GL_TRIANGLE_FAN, None, 
    ("v2i", point_list),
    ("c3B", [random.randrange(255)]*(3*ec)))

This causes this Polygon triangulation:

---

For any polygon you want to draw you've to find a proper Polygon triangulation. The triangulation can vary and one of the Triangle primitive types GL_TRIANGLES, GL_TRIANGLE_STRIP or GL_TRIANGLE_FAN has to be used.

In some cases it is sufficient to change the start point of the polygon and to use the primitive type GL_TRIANGLE_FAN.

e.g. the point list (from the commnet) [488, 485, 375, 73, 61, 48, 70, 257, 119, 260, 418, 327] can be change to [375, 73, 61, 48, 70, 257, 119, 260, 418, 327, 488, 485]:

---

For a more general approach you've to implement Polygon triangulation algorithm like the Two ears theorem, but OpenGL doesn't do the job for you.

---

Related Topics