It is very convenient when I play mobile game and watch video at the
same time.
Tested on Linux mint Cinnamon as well as Windows 10.
Signed-off-by: Yu-Chen Lin <npes87184@gmail.com>
This partially reverts commit f00c6c5b13.
On Ctrl+C, we need to execute cleanup code. For instance, if recording
is enabled, we need to write MP4 file trailer on exit.
Custom SDL signal handlers were disabled because it leaded to process
hanging on Ctrl+C during network calls on initialization, but now it
seems to work correctly, the network calls return immediately on signal.
In practice, proc_show_touches may not be used uninitialized, since it
checks the flag options->show_touches, but the compiler can't know that,
so initialize it to avoid the warning.
Enabling "show touches" involves the execution of an adb command, which
takes some time.
In order to parallelize, execute the command as soon as possible, but
reap the process only once everything is initialized.
If --show-touches is set, then the option must be disabled on quit.
Since it executes an adb command, it takes some time, so close the
window beforehand so that the close window button does not seem
unresponsive.
Request SDL not to replace the SIGINT and SIGTERM handlers, so that the
process is immediately terminated on Ctrl+C.
This avoids process hanging on Ctrl+C during network calls on
initialization.
Some of them accepted a timeout, but it was not used since
commit 9b056f5091 anymore.
On Windows and MacOS, resizing blocks the event loop, so resizing events
are not triggered:
- <https://bugzilla.libsdl.org/show_bug.cgi?id=2077>
- <https://stackoverflow.com/a/40693139/1987178>
As a workaround, register an event watcher to render the screen from
another thread.
Since the whole event loop is blocked during resizing, the screen
content is not refreshed (on Windows and MacOS) until resizing ends.
The serial is needed for many server actions, but this is an
implementation detail, so the caller should not have to provide it on
every call.
Instead, store the serial in the server instance on server_start().
This paves the way to implement the "adb forward" fallback properly.
SDL_net is not very suitable for scrcpy.
For example, SDLNet_TCP_Accept() is non-blocking, so we have to wrap it
by calling many SDL_Net-specific functions to make it blocking.
But above all, SDLNet_TCP_Open() is a server socket only when no IP is
provided; otherwise, it's a client socket. Therefore, it is not possible
to create a server socket bound to localhost, so it accepts connections
from anywhere.
This is a problem for scrcpy, because on start, the application listens
for nearly 1 second until it accepts the first connection, supposedly
from the device. If someone on the local network manages to connect to
the server socket first, then they can stream arbitrary H.264 video.
This may be troublesome, for example during a public presentation ;-)
Provide our own simplified API (net.h) instead, implemented for the
different platforms.
The "screen control" handled user input, which happened to be only
used to control the screen.
The controller and screen were passed to every function. Instead, group
them in a struct input_manager.
The purpose is to add a new shortcut to enable/disable FPS counter. This
feature is not related to "screen control", and will require access to
the "frames" instance.
On startup, the client has to:
1. listen on a port
2. push and start the server to the device
3. wait for the server to connect (accept)
4. read device name and size
5. initialize SDL
6. initialize the window and renderer
7. show the window
From the execution of the app_process command to start the server on the
device, to the execution of the java main method, it takes ~800ms. As a
consequence, step 3 also takes ~800ms on the client.
Once complete, the client initializes SDL, which takes ~500ms.
These two expensive actions are executed sequentially:
HOST DEVICE
listen on port | |
push/start the server |----------------->|| app_process loads the jar
accept the connection . ^ ||
. | ||
. | WASTE ||
. | OF ||
. | TIME ||
. | ||
. | ||
. v X execution of our java main
connection accepted |<-----------------| connect to the host
init SDL || |
|| ,----------------| send frames
|| |,---------------|
|| ||,--------------|
|| |||,-------------|
|| ||||,------------|
init window/renderer | |||||,-----------|
display frames |<++++++-----------|
(many frames skipped)
The rationale for step 3 occuring before step 5 is that initializing
SDL replaces the SIGTERM handler to receive the event in the event loop,
so pressing Ctrl+C during step 5 would not work (since it blocks the
event loop).
But this is not so important; let's parallelize the SDL initialization
with the app_process execution (we'll just add a timeout to the
connection):
HOST DEVICE
listen on port | |
push/start the server |----------------->||app_process loads the jar
init SDL || ||
|| ||
|| ||
|| ||
|| ||
|| ||
accept the connection . ||
. X execution of our java main
connection accepted |<-----------------| connect to the host
init window/renderer | |
display frames |<-----------------| send frames
|<-----------------|
In addition, show the window only once the first frame is available to
avoid flickering (opening a black window for 100~200ms).
Note: the window and renderer are initialized after the connection is
accepted because they use the device information received from the
device.
SDLNet_TCP_Close() not only closes, but also release the resources.
Therefore, we must not close the socket if another thread attempts to
read it.
For that purpose, move socket closing from server_stop() to
server_destroy().
Replace screen_update() by a higher-level screen_update_frame() handling
the whole frame updating, so that scrcpy.c just call it without managing
implementation details.
Expose frames_offer_decoded_frame() and frames_consume_rendered_frame()
so that callers are not exposed to frame swapping (between the decoding
and rendering frames) details.
The skip_frames flag was a non-configurable runtime flag. Since it is
not exposed to the user, there is no need for a (possible) runtime cost.
For testing purpose, we still want it to be configurable, so make it a
compilation flag.
We encounter some problems with SDL2_image on MSYS2 (Windows), so
implement our own XPM parsing which does not depend on SDL_image.
The input XPM is considered safe (it's in our source repo), so do not
check XPM format errors. This implies that read_xpm() is not safe to
call on any unsafe input.
Although less straightforward, use SDL_CreateRGBSurfaceFrom() instead of
SDL_CreateRGBSurfaceWithFormatFrom() because it is available with SDL
versions older than 2.0.5.
Yu-Chen Lin
Romain Vimont
Ivan Gorinov
Philipp Sandhaus
Grief
Adonis Najimi
Romain Vimont