Caputre Opengl Window in X11 with Fast Framerate - Possible

Capturing video out of an OpenGL window in Windows

There are two different questions here - how to grab frames from an OpenGL application, and how to turn them into a movie file.

The first question is easy enough; you just grab each frame with glReadPixels() (via a PBO if you need the performance).

The second question is a little harder since the cross-platform solutions (ffmpeg) tend to be GPL'd or LGPL'd. Is LGPL acceptable for your project? The Windows way of doing this (DirectShow) is a bit of a headache to use.

Edit: Since LGPL is ok and you can use ffmpeg, see here for an example of how to encode video.

setting max frames per second in openGL

You have two different ways to solve this problem:

Suppose that you have a variable called maximum_fps, which contains for the maximum number of frames you want to display.
Then You measure the amount of time spent on the last frame (a timer will do)
Now suppose that you said that you wanted a maximum of 60FPS on your application. Then you want that the time measured be no lower than 1/60. If the time measured s lower, then you call sleep() to reach the amount of time left for a frame.
Or you can have a variable called tick, that contains the current "game time" of the application. With the same timer, you will incremented it at each main loop of your application. Then, on your drawing routines you calculate the positions based on the tick var, since it contains the current time of the application.
The big advantage of option 2 is that your application will be much easier to debug, since you can play around with the tick variable, go forward and back in time whenever you want. This is a big plus.

Save OpenGL Rendering to Video

There most certainly are great video capture software out there you could use to capture your screen, even when running a full screen OpenGL game.

If you are using new versions of OpenGL, as genpfault has mentioned you can use PBOs. If you are using legacy OpenGL (version 1.x), here's how you can capture the screen:

glFinish(); // Make sure everything is drawn
glReadBuffer(GL_FRONT);
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glPixelStorei(GL_PACK_ROW_LENGTH, 0);
glPixelStorei(GL_PACK_SKIP_ROWS, 0);
glPixelStorei(GL_PACK_SKIP_PIXELS, 0);
glReadPixels(blx, bly, w, h, mode, GL_UNSIGNED_BYTE, GL_BGRA);

where blx and bly are the bottom left coordinates of the part of the screen you want to capture (in your case (0, 0)) and w and h are the width and height of the box to be captured. See the reference for glReadPixels for more info, such as the last parameter.

Writing captured screen (at your desired rate, for example 24 fps) to a video file is a simple matter of choosing the file format you want (for example raw video), write the header of the video and write the images (image by image if raw, or image differences in some other format etc)

Linux: Screen desktop video capture over network, and VNC framerate

You should get a badge for such a long well though out question. ;-)

In answer to your primary question, VNC uses the RFB protocol which is a remote frame buffer protocol (thus the acronym) not a streaming video protocol. The VNC client sends a FrameBufferUpdateRequest message to the server which contains a viewport region that the client is interested in and an incremental flag. If the incremental flag is not set then the server will respond with a FrameBufferUpdate message that contains the content of the region requested. If the incremental flag is set then the server may respond with a FrameBufferUpdate message that contains whatever parts of the region requested that have changed since the last time the client was sent that region.

The definition of how requests and updates interact is not crisply defined. The server won't necessarily respond to every request with an update if nothing has changed. If the server has multiple requests queued from the client it is also allowed to send a single update in response. In addition, the client really needs to be able to respond to an asynchronous update message from the server (not in response to a request) otherwise the client will fall out of sync (because RFB is not a framed protocol).

Often clients are simply implemented to send incremental update requests for the entire frame buffer viewport at a periodic interval and handle any server update messages as they arrive (i.e. no attempt is made to tie requests and updates together).

Here is a description of FrameBufferUpdateRequest messages.

How to take a screenshot of desktop fast with Java in Windows (ffmpeg, etc.)?

Using the built-in Robots class is way easier than other Java libraries and should probably fit your needs.

If you need a smooth video with >= 30fps (more than 30 screenshots per second), you should first try the Robots approach plus performance improvements there using asynchronous storing of the screenshots.

If it doesn't work for you, try using JNA and that is (even though it's more complex) almost guaranteed to work for smooth screen capturing.

Approach with Robots

The robots class is indeed capable of doing what you want, the problem most screen capturing approaches with Robots have is the saving of the screenshots. An approach could look like that: Looping over the captureScreen() method, grabbing the screen into a BufferedImage, convert it to a byte array and save it with an asynchronous file writer to a target file after adding the future reference of your image to the ArrayList to be able to keep going while storing the image data.

// Pseudo code
while (capturing)
{
    grab bufferedImage (screenCapture) from screen
    convert bufferImage to byte array
    start asynchronous file channel to write to the output file
      and add the future reference (return value) to the ArrayList
}

Approach with JNA

Original Question:
How to take screenshots fast in Java?

As it is bad practice to just link, I will post the example here:

import java.awt.Rectangle;
import java.awt.image.BufferedImage;
import java.awt.image.ColorModel;
import java.awt.image.DataBuffer;
import java.awt.image.DataBufferInt;
import java.awt.image.DataBufferUShort;
import java.awt.image.DirectColorModel;
import java.awt.image.Raster;
import java.awt.image.WritableRaster;

import com.sun.jna.Native;
import com.sun.jna.platform.win32.W32API;
import com.sun.jna.win32.W32APIOptions;

public class JNAScreenShot
{

    public static BufferedImage getScreenshot(Rectangle bounds)
    {
        W32API.HDC windowDC = GDI.GetDC(USER.GetDesktopWindow());
        W32API.HBITMAP outputBitmap = GDI.CreateCompatibleBitmap(windowDC, bounds.width, bounds.height);
        try
        {
            W32API.HDC blitDC = GDI.CreateCompatibleDC(windowDC);
            try
            {
                W32API.HANDLE oldBitmap = GDI.SelectObject(blitDC, outputBitmap);
                try
                {
                    GDI.BitBlt(blitDC, 0, 0, bounds.width, bounds.height, windowDC, bounds.x, bounds.y, GDI32.SRCCOPY);
                }
                finally
                {
                    GDI.SelectObject(blitDC, oldBitmap);
                }
                GDI32.BITMAPINFO bi = new GDI32.BITMAPINFO(40);
                bi.bmiHeader.biSize = 40;
                boolean ok = GDI.GetDIBits(blitDC, outputBitmap, 0, bounds.height, (byte[]) null, bi, GDI32.DIB_RGB_COLORS);
                if (ok)
                {
                    GDI32.BITMAPINFOHEADER bih = bi.bmiHeader;
                    bih.biHeight = -Math.abs(bih.biHeight);
                    bi.bmiHeader.biCompression = 0;
                    return bufferedImageFromBitmap(blitDC, outputBitmap, bi);
                }
                else
                {
                    return null;
                }
            }
            finally
            {
                GDI.DeleteObject(blitDC);
            }
        }
        finally
        {
            GDI.DeleteObject(outputBitmap);
        }
    }

    private static BufferedImage bufferedImageFromBitmap(GDI32.HDC blitDC, GDI32.HBITMAP outputBitmap, GDI32.BITMAPINFO bi)
    {
        GDI32.BITMAPINFOHEADER bih = bi.bmiHeader;
        int height = Math.abs(bih.biHeight);
        final ColorModel cm;
        final DataBuffer buffer;
        final WritableRaster raster;
        int strideBits = (bih.biWidth * bih.biBitCount);
        int strideBytesAligned = (((strideBits - 1) | 0x1F) + 1) >> 3;
        final int strideElementsAligned;
        switch (bih.biBitCount)
        {
            case 16:
                strideElementsAligned = strideBytesAligned / 2;
                cm = new DirectColorModel(16, 0x7C00, 0x3E0, 0x1F);
                buffer = new DataBufferUShort(strideElementsAligned * height);
                raster = Raster.createPackedRaster(buffer, bih.biWidth, height, strideElementsAligned, ((DirectColorModel) cm).getMasks(), null);
                break;
            case 32:
                strideElementsAligned = strideBytesAligned / 4;
                cm = new DirectColorModel(32, 0xFF0000, 0xFF00, 0xFF);
                buffer = new DataBufferInt(strideElementsAligned * height);
                raster = Raster.createPackedRaster(buffer, bih.biWidth, height, strideElementsAligned, ((DirectColorModel) cm).getMasks(), null);
                break;
            default:
                throw new IllegalArgumentException("Unsupported bit count: " + bih.biBitCount);
        }
        final boolean ok;
        switch (buffer.getDataType())
        {
            case DataBuffer.TYPE_INT:
            {
                int[] pixels = ((DataBufferInt) buffer).getData();
                ok = GDI.GetDIBits(blitDC, outputBitmap, 0, raster.getHeight(), pixels, bi, 0);
            }
                break;
            case DataBuffer.TYPE_USHORT:
            {
                short[] pixels = ((DataBufferUShort) buffer).getData();
                ok = GDI.GetDIBits(blitDC, outputBitmap, 0, raster.getHeight(), pixels, bi, 0);
            }
                break;
            default:
                throw new AssertionError("Unexpected buffer element type: " + buffer.getDataType());
        }
        if (ok)
        {
            return new BufferedImage(cm, raster, false, null);
        }
        else
        {
            return null;
        }
    }

    private static final User32 USER = User32.INSTANCE;

    private static final GDI32 GDI = GDI32.INSTANCE;

}

interface GDI32 extends com.sun.jna.platform.win32.GDI32
{
    GDI32 INSTANCE = (GDI32) Native.loadLibrary(GDI32.class);

    boolean BitBlt(HDC hdcDest, int nXDest, int nYDest, int nWidth, int nHeight, HDC hdcSrc, int nXSrc, int nYSrc, int dwRop);

    HDC GetDC(HWND hWnd);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, byte[] pixels, BITMAPINFO bi, int usage);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, short[] pixels, BITMAPINFO bi, int usage);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, int[] pixels, BITMAPINFO bi, int usage);

    int SRCCOPY = 0xCC0020;
}

interface User32 extends com.sun.jna.platform.win32.User32
{
    User32 INSTANCE = (User32) Native.loadLibrary(User32.class, W32APIOptions.UNICODE_OPTIONS);

    HWND GetDesktopWindow();
}

More information and approaches

Increasing screen capture speed when using Java and awt.Robot
http://www.dreamincode.net/forums/topic/234896-faster-screen-capture/
How to get over 30FPS using Java in a Screen Capture Program?
http://ffmpeg.org

See also

http://www.thepcwizard.in/2012/12/java-screen-capturing-tutorial.html
How to develop screen capture to video application
http://www.javalobby.org/forums/thread.jspa?threadID=16400&tstart=0
http://hiddensciencex.blogspot.co.at/2014/01/fast-screen-capture-in-java-example.html
http://www.coderanch.com/t/340180/GUI/java/efficient-screenshot-Java
http://www.javaworld.com/article/2071755/learn-java/capture-the-screen.html
ffmpeg for screen capture?
Java applet screen capture to a video
Screen Capture of DirectX programs with Java

What is hardware cursor and how does it work?

Hardware Cursor means, that the GPU provides to draw a (small) overlay picture over the screen framebuffer, which position can be changed by two registers (or so) on the GPU. So moving around the pointer doesn't require to redraw the portions of the framebuffer that were previously obstructed.

Relation to OpenGL: None!