Create an On-Screen Keyboard

How do I create an on screen keyboard in java?

You could construct the buttons through the use of for loops. One loop for every keyboard row is a plausible approach.

String row1 = "1234567890";
String row2 = "qwertyuiop";
// and so forth
String[] rows = { row1, row2, .. };
for (int i = 0; i < rows.length; i++) {
    char[] keys = rows[i].toCharArray();
    for (int j = 0; i < keys.length; j++) {
        JButton button = new JButton(Character.toString(keys[j]));
        // add button
    }
}
// add special buttons like space bar

This could be done more elegantly through a more OOP approach, but this basic loop system will work.

How to create an on-screen keyboard which will show which keys are being typed currently?

Try extending this example

jQuery:

$(function () {
  $("#input").keydown(function (event) {
      $("#" + String.fromCharCode(event.which)).addClass("trigger");
  }).keyup(function (event) {
      $("#" + String.fromCharCode(event.which)).removeClass("trigger");
  });
});​

You can create a keyboard using pure HTML and CSS. See this beautiful example

Better solution for making on-screen keyboard

Why you even needs to write code? As a better and more logical solution you can use the on-screen keyboard application that comes with windows for this purpose:

System.Diagnostics.Process.Start("osk.exe");

How does on-screen (virtual) keyboard works in Win10

SendInput is the preferred method to generate user input in software. The Windows on-screen keyboard probably uses it for everything except Ctrl+Alt+Delete which I believe has some kind of special handling. The on-screen keyboard is only able to generate Ctrl+Alt+Delete in certain configurations.

Software-generated input is merged with normal hardware input in the RIT (Raw Input Thread) in the kernel.

A low-level keyboard hook can detect software-generated input.

Create an On-screen Keyboard

You must compromise:

If you want to simulate keyboard input, then you must use SendInput, which means being at the mercy of which window currently has focus. SendInput is like hitting the keys on your physical keyboard. The only way you can send your keystrokes to a specific window using your keyboard is to ALT+TAB to the right window.

If you want to send keystrokes to a specific window, then you incur funky behavior:

1. Applications handle input differently. And simple WM_KEYDOWN / WM_KEYUP messages are not the only way to detect keyboard input. For example there is also the keyboard state (GetKeyboardState()) which you will have a harder time simulating. This is most likely what you're experiencing.
2. Applications may RELY on the standard behavior of having focus while receiving keyboard input messages. By posting messages to these applications, you invoke strange out-of-order behavior that may crash them.
3. Now multiple windows on the system can be receiving keyboard input at the same time. This might also cause strange behavior.
4. (etc...) Hooks won't be called for this input, your keyboard / input drivers won't see it, it won't be recognized by things like DirectInput... basically it's a never-ending patchwork of issues by doing something the bad-bear way.

There is no way around those side-effects; it's the consequence of doing shady stuff.

---

A solution for your purposes, because you're targeting a single specific application, may be to use PostMessage in conjunction with SetKeyboardState to simulate the keyboard state including shift positions.

How do I display a keyboard on-screen?

One can do this by using the python library keyboardlayout.
Using it one can show an qwerty or azerty layout and this example highlights specifc keys by passing in the overrides argument. It works with pygame and tkinter.

Here's how you use it:

1. Decide which layout you want to show, qwerty or azerty_laptop. That is the layout_name input
2. Decide how big you want your keyboard to be. You control the size by setting the size of a single letter key, and that size (and a padding input) determines the whole keyboard size. The size input for a key is letter_key_size and it needs you to pass in (width_size_px, height_size_px).
3. Decide what settings you want for the keyboard. Specifically, where (x, y) do you want it, what color should it be, what padding should it have in pixels? All of those settings are stored in the KeyboardInfo class instance. Remember that in pygame, (0, 0) is the top left and x increases to the right and y increases moving downward.
4. Decide what settings you want to use display the keys, specifically: margin between keys in pixels, background color, text color, font, and padding from the key edges to the text in pixels. All of that info is stored in the KeyInfo class instance.
5. Once you instantiate KeyInfo, KeyboardInfo, and set letter_key_size and layout_name you use that information to instantiate an actual KeyboardLayout class instance. That instance contains the keyboard image that you want to draw. It's also a pygame.sprite.Group so to display it we use the normal pygame method sprite_group.draw(screen).

Gathering that all together and putting it into action we get:

import keyboardlayout as kl
import pygame

layout_name = 'qwerty'
pygame.init()

# set the letter key size in pixels
key_size = 60
grey = pygame.Color('grey')
# set the keyboard position and color info
keyboard_info = kl.KeyboardInfo(
    position=(0, 0),
    padding=2,
    color=~grey
)
# set the letter key color, padding, and margin info in px
key_info = kl.KeyInfo(
    margin=10,
    color=grey,
    txt_color=~grey,  # invert grey
    txt_font=pygame.font.SysFont('Arial', key_size//4),
    txt_padding=(key_size//6, key_size//10)
)
# set the letter key size info in px
letter_key_size = (key_size, key_size)  # width, height
keyboard_layout = kl.KeyboardLayout(
    layout_name,
    keyboard_info,
    letter_key_size,
    key_info
)
# set the pygame window to the size of the keyboard
screen = pygame.display.set_mode(
    (keyboard_layout.rect.width, keyboard_layout.rect.height))
screen.fill(pygame.Color('black'))

# draw the keyboard on the pygame screen
keyboard_layout.draw(screen)
pygame.display.update()

# loop until the user closes the pygame window
running = True
while running:
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            pygame.display.quit()
            running = False

pygame.quit()

Here are some samples of what it can do:

---

Related Topics