On Windows and MacOS, resizing blocks the event loop, so resizing events
are not triggered:
- <https://bugzilla.libsdl.org/show_bug.cgi?id=2077>
- <https://stackoverflow.com/a/40693139/1987178>
As a workaround, register an event watcher to render the screen from
another thread.
Since the whole event loop is blocked during resizing, the screen
content is not refreshed (on Windows and MacOS) until resizing ends.
"adb reverse" currently does not work over tcpip (i.e. on a device
connected by "adb connect"):
<https://issuetracker.google.com/issues/37066218>
To work around the problem, if the call to "adb reverse" fails, then
fallback to "adb forward", and reverse the client/server roles.
Keep the "adb reverse" mode as the default because it does not involve
connection retries: when using "adb forward", the client must try to
connect successively until the server listens.
Due to the tunnel, every connect() will succeed, so the client must
attempt to read() to detect a connection failure. For this purpose, when
using the "adb forward" mode, the server initially writes a dummy byte,
read by the client.
Fixes <https://github.com/Genymobile/scrcpy/issues/5>.
The serial is needed for many server actions, but this is an
implementation detail, so the caller should not have to provide it on
every call.
Instead, store the serial in the server instance on server_start().
This paves the way to implement the "adb forward" fallback properly.
The SDL mouse wheel event seems inconsistent between horizontal and
vertical scrolling.
> Movements to the left generate negative x values and to the right
> generate positive x values. Movements down (scroll backward) generate
> negative y values and up (scroll forward) generate positive y values.
<https://wiki.libsdl.org/SDL_MouseWheelEvent#Remarks>
Reverse the horizontal.
Fixes <https://github.com/Genymobile/scrcpy/issues/49>.
The text input control_event was initially designed for mapping
SDL_TextInputEvent, limited to 32 characters.
For simplicity, the copy/paste feature was implemented using the same
control_event: it just sends the text to paste.
However, the pasted text might have a length breaking some assumptions:
- on the client, the event max-size was smaller than the text
max-length,
- on the server, the raw buffer storing the events was smaller than the
max event size.
Fix these inconsistencies, and encode the length on 2 bytes, to accept
more than 256 characters.
Fixes <https://github.com/Genymobile/scrcpy/issues/10>.
Paste computer clipboard to the device on Ctrl+v.
The other direction (pasting the device clipboard to the computer) is
not implemented. It would require a communication channel from the
device to the computer, other than the socket used by the video stream.
The High DPI support is enabled by default, so that the renderer use the
full definition of High DPI screens.
However, there are still mouse coordinates problems on some MacOS having
High DPI support (but not all), so expose a way to disable it.
The decoder sometimes returned a non-zero value on error, but not on
every path.
Since we never use the value, always return 0 at the end (like in the
controller).
SDL_MouseWheelEvent does not provide the mouse location, so we used
SDL_GetMouseState() to retrieve it.
Unfortunately, SDL_GetMouseState() returns a position expressed in the
window coordinate system while the position filled in SDL events are
expressed in the renderer coordinate system. As a consequence, the
scroll was not applied at the right position on the device.
Therefore, convert the coordinate system.
See <https://stackoverflow.com/questions/49111054/how-to-get-mouse-position-on-mouse-wheel-event>.
Use high DPI if available.
Note that on Mac OS X, setting this flag is not sufficient:
> On Apple's OS X you must set the NSHighResolutionCapable Info.plist
> property to YES, otherwise you will not receive a High DPI OpenGL
> display.
<https://wiki.libsdl.org/SDL_CreateWindow#flags>
On user request to quit, two kinds of blocking calls must be interrupted
on the server:
1. the reads from and writes to the socket;
2. the call to MediaCodec.dequeueOutputBuffer().
The former case was handled by calling shutdown() on the socket from the
client, but the latter was not managed.
There is no easy way to wake this call properly, so just terminate the
process from the client (i.e. send SIGTERM on Linux) instead.
The server is copied to /data/local/tmp/scrcpy-server.jar and executed
on the device.
As soon as we are connected, we can unlink (rm) it from /data/local/tmp,
to keep the device clean.
The server is currently a JAR, but it may ba an APK or a DEX, so the
variable name should not contain the type.
Rename the environment variable, the Meson options and the C
definitions.
Expose net_recv_all() and net_send_all(), equivalent of net_recv() and
net_send(), but that waits/retries until the requested length has been
transferred.
Use these new functions where it was (wrongly) assumed that the
requested length had been transferred.
SDL_net is not very suitable for scrcpy.
For example, SDLNet_TCP_Accept() is non-blocking, so we have to wrap it
by calling many SDL_Net-specific functions to make it blocking.
But above all, SDLNet_TCP_Open() is a server socket only when no IP is
provided; otherwise, it's a client socket. Therefore, it is not possible
to create a server socket bound to localhost, so it accepts connections
from anywhere.
This is a problem for scrcpy, because on start, the application listens
for nearly 1 second until it accepts the first connection, supposedly
from the device. If someone on the local network manages to connect to
the server socket first, then they can stream arbitrary H.264 video.
This may be troublesome, for example during a public presentation ;-)
Provide our own simplified API (net.h) instead, implemented for the
different platforms.
The syntax was correct, but less readable, and it unnecessarily zeroed
the fields other than "type".
Create the event properly, from a separate method.
screen_render() should not be called on initialization:
1. it is useless, since the window is hidden until the first frame;
2. it writes an empty texture (probably green) to the renderer.
The "screen control" handled user input, which happened to be only
used to control the screen.
The controller and screen were passed to every function. Instead, group
them in a struct input_manager.
The purpose is to add a new shortcut to enable/disable FPS counter. This
feature is not related to "screen control", and will require access to
the "frames" instance.
The device serial was provided as a positional argument:
scrcpy 0123456789abcdef
Instead, expose it as an optional argument, -s or --serial:
scrcpy -s 0123456789abcdef
This avoids inconsistency between platforms when the positional
argument is passed before the options (which is undefined).
Since Meson 0.44, subproject_dir may not be '.' anymore. This implies we
must move app/ and server/ to a subprojects/ directory, which requires
to also change some gradle files.
Instead, just use subdir(), with options to disable building of the app
or the server.
The client was built with Meson, the server with Gradle, and were run by
a Makefile.
Add a Meson script for the server (which delegates to Gradle), and a
parent script to build and install both the client and the server to the
system, typically with:
meson --buildtype release build
cd build
ninja
sudo ninja install
In addition, use a separate Makefile to build a "portable" version of
the application (where the client expects the server to be in the
current directory). Typically:
make release-portable
cd dist/scrcpy
./scrcpy
This is especially useful for Windows builds, which are not "installed".
The current meson version is able to generate a config.h from a
configuration data object without any template.
However, older versions of meson require a template, so provide it for
compatibility.
The SDL clean up does not crash anymore on exit, probably since the
memory corruption caused by calling SDLNet_TCP_Close() too early has
been resolved.
On startup, the client has to:
1. listen on a port
2. push and start the server to the device
3. wait for the server to connect (accept)
4. read device name and size
5. initialize SDL
6. initialize the window and renderer
7. show the window
From the execution of the app_process command to start the server on the
device, to the execution of the java main method, it takes ~800ms. As a
consequence, step 3 also takes ~800ms on the client.
Once complete, the client initializes SDL, which takes ~500ms.
These two expensive actions are executed sequentially:
HOST DEVICE
listen on port | |
push/start the server |----------------->|| app_process loads the jar
accept the connection . ^ ||
. | ||
. | WASTE ||
. | OF ||
. | TIME ||
. | ||
. | ||
. v X execution of our java main
connection accepted |<-----------------| connect to the host
init SDL || |
|| ,----------------| send frames
|| |,---------------|
|| ||,--------------|
|| |||,-------------|
|| ||||,------------|
init window/renderer | |||||,-----------|
display frames |<++++++-----------|
(many frames skipped)
The rationale for step 3 occuring before step 5 is that initializing
SDL replaces the SIGTERM handler to receive the event in the event loop,
so pressing Ctrl+C during step 5 would not work (since it blocks the
event loop).
But this is not so important; let's parallelize the SDL initialization
with the app_process execution (we'll just add a timeout to the
connection):
HOST DEVICE
listen on port | |
push/start the server |----------------->||app_process loads the jar
init SDL || ||
|| ||
|| ||
|| ||
|| ||
|| ||
accept the connection . ||
. X execution of our java main
connection accepted |<-----------------| connect to the host
init window/renderer | |
display frames |<-----------------| send frames
|<-----------------|
In addition, show the window only once the first frame is available to
avoid flickering (opening a black window for 100~200ms).
Note: the window and renderer are initialized after the connection is
accepted because they use the device information received from the
device.
SDLNet_TCP_Close() not only closes, but also release the resources.
Therefore, we must not close the socket if another thread attempts to
read it.
For that purpose, move socket closing from server_stop() to
server_destroy().
Replace screen_update() by a higher-level screen_update_frame() handling
the whole frame updating, so that scrcpy.c just call it without managing
implementation details.
Do not wait 100ms anymore to let the server print any exception: we
justly want to ignore them.
Moreover, there is no nanosleep() on Windows, so this solve another
problem.
When the video stream socket is closed and read_packey() returns -1,
av_read_frame() still returns 0.
To detect EOF, check the flag eof_reached in the AVIOContext.
This avoids garbage errors on closing.
Expose frames_offer_decoded_frame() and frames_consume_rendered_frame()
so that callers are not exposed to frame swapping (between the decoding
and rendering frames) details.
The skip_frames flag was a non-configurable runtime flag. Since it is
not exposed to the user, there is no need for a (possible) runtime cost.
For testing purpose, we still want it to be configurable, so make it a
compilation flag.
We encounter some problems with SDL2_image on MSYS2 (Windows), so
implement our own XPM parsing which does not depend on SDL_image.
The input XPM is considered safe (it's in our source repo), so do not
check XPM format errors. This implies that read_xpm() is not safe to
call on any unsafe input.
Although less straightforward, use SDL_CreateRGBSurfaceFrom() instead of
SDL_CreateRGBSurfaceWithFormatFrom() because it is available with SDL
versions older than 2.0.5.
The purpose of handle_shortcut() was to group all actions together,
whether they are initiated from a text input event or a keycode event.
However, it did not handle the case where it was initiated from a mouse
event (a right-click must turn the screen on), since the action was
identified by the shortcut char.
Instead, expose one function per action, to be called directly from
where the event is handled.
The right-click is almost useless on Android, so use it to turn the
screen on.
Add a new control event type (command) to request the server to turn the
screen on.
Romain Vimont