The video screen size on the client may differ from the real device
screen size (e.g. the video stream may be scaled down). As a
consequence, mouse events must be scaled to match the real device
coordinates.
For this purpose, make the client send the video screen size along with
the absolute pointer location, and the server scale the location to
match the real device size before injecting mouse events.
Ctrl, Alt, Shift and Meta should not be transmitted to the Android
device: they may generate unwanted events. For instance, resizing the
window using Alt+click will generate an Alt event which may open a menu
on the device.
All keycodes that generate a text input must also be excluded, to avoid
the text input to be written twice.
To control the device from the computer:
- retrieve mouse and keyboard SDL events;
- convert them to Android events;
- serialize them;
- send them on the same socket used by the video stream (but in the
opposite direction);
- deserialize the events on the Android side;
- inject them using the InputManager.
Romain Vimont