CI and the Linux display

Electron applications and VS Code open real windows, so the tests need a display. On Windows and macOS they run on the desktop. On Linux, you choose:

Hidden on a virtual X server, Xvfb: for CI, and for local runs that should leave your desktop alone.
Visible in a separate Xephyr window, which shows the run apart from your other windows.
On your normal desktop.

Display profiles

RobotCode’s robot.toml can run the tests through a wrapper command, and each profile can have its own. The example project defines three display profiles, which this repository uses as well, and a profile small-screen that changes their screen size:

# Robot Framework configuration, used by RobotCode (https://robotcode.io).
# Run the tests from this folder with `robotcode robot`.
# Personal settings, such as a local VSCODE_EXECUTABLE, belong in .robot.toml, which is not committed.

paths = ["tests"]
output-dir = "results"

# The locators are variables of the resources in tests/resources. When another VS Code version,
# or a fork, needs other locators, add a profile for it that overrides them, usually together
# with VSCODE_VERSION, and select it with `robotcode -p <profile> robot`. The keywords stay the same.
#
# This profile overrides the command palette locator with an equivalent selector, to show how.
[profiles.locator-override]
description = "Find command palette rows through the quick input widget instead of its list"
extend-variables = { COMMAND_PALETTE_ROW = '.quick-input-widget .monaco-list-row[aria-label="{title}"], .quick-input-widget .monaco-list-row[aria-label^="{title}, "]' }

# Where the tests run. Without a display profile they run on the normal desktop.
# The wrappers run as shell scripts; RobotCode appends its command line, which they run as "$@".
# SCREEN_SIZE sets the screen size of xvfb and xephyr, default 1920x1080, from the shell or a profile's extend-env.
# Both start the window manager Openbox, if it is installed, so that windows can be maximised.
[profiles.xvfb]
description = "Run hidden on a Full HD Xvfb screen (Linux)"
enabled.if = "platform.system() == 'Linux'"
wrapper = ["sh", "-c", '''
env -u WAYLAND_DISPLAY XDG_SESSION_TYPE=x11 xvfb-run -a -s "-screen 0 ${SCREEN_SIZE:-1920x1080}x24" \
    sh -c 'command -v openbox >/dev/null && openbox >/dev/null 2>&1 & exec "$@"' wm "$@"
''', "xvfb-run"]

[profiles.xephyr]
description = "Run in a separate Full HD Xephyr window on the desktop (Linux)"
enabled.if = "platform.system() == 'Linux'"
# Starts Xephyr on a free display, runs the tests on it, and ends it afterwards.
wrapper = ["sh", "-c", '''
fifo=$(mktemp -u) && mkfifo "$fifo" || exit 1
Xephyr -displayfd 3 -screen "${SCREEN_SIZE:-1920x1080}" -title "Robot Framework" -noreset 3>"$fifo" &
xephyr=$!
trap 'kill "$xephyr" 2>/dev/null; rm -f "$fifo"' EXIT
read -r display <"$fifo" || exit 1
command -v openbox >/dev/null && DISPLAY=":$display" openbox >/dev/null 2>&1 &
env -u WAYLAND_DISPLAY XDG_SESSION_TYPE=x11 DISPLAY=":$display" "$@"
''', "xephyr-run"]

[profiles.local]
description = "Run on the normal desktop"

# Changes the screen size of xvfb and xephyr. extend-env keeps the variables that other selected profiles set,
# where env would replace them. Combine it with xvfb or xephyr, for example `robotcode -p xvfb -p small-screen robot`.
[profiles.small-screen]
description = "Use a 1280x800 screen with xvfb or xephyr (Linux)"
enabled.if = "platform.system() == 'Linux'"
extend-env = { SCREEN_SIZE = "1280x800" }

Choose one with -p, and combine it with other profiles:

robotcode -p xvfb robot                          # hidden on a Full HD Xvfb screen
robotcode -p xephyr robot                        # in a separate Full HD Xephyr window
robotcode -p local robot                         # on the normal desktop
robotcode -p xvfb -p small-screen robot          # hidden, on a 1280×800 screen
robotcode -p xvfb -p locator-override robot      # hidden, with another profile

Linux only: xvfb and xephyr are switched on only on Linux, through enabled.if. Without a display profile, the tests run on the desktop.
Wayland: on a Wayland desktop, Electron applications and VS Code follow WAYLAND_DISPLAY and XDG_SESSION_TYPE, and would open their windows on the desktop even under Xvfb. Both wrappers therefore remove WAYLAND_DISPLAY and set the session type to X11. env in robot.toml cannot remove a variable, which is why env -u sits in the wrapper.
Screen size: SCREEN_SIZE sets the screen of both profiles, default 1920x1080. Set it in a profile with extend-env, as small-screen does, or in the shell, for example SCREEN_SIZE=1280x800 robotcode -p xvfb robot. extend-env keeps the variables that other selected profiles set, where env would replace them.
Window manager: both profiles start the window manager Openbox on their display, if it is installed, so that windows can be maximised there. Without it, the tests still run, with windows at their default size.
Packages: xvfb needs Xvfb, from the package xvfb on Debian and Ubuntu or xorg-server-xvfb on Arch Linux. xephyr needs Xephyr, from xserver-xephyr or xorg-server-xephyr. Openbox comes from the package openbox.

Without RobotCode

With plain robot, put xvfb-run in front. On a Wayland desktop, also remove WAYLAND_DISPLAY and set the session type to X11:

xvfb-run -a -s "-screen 0 1920x1080x24" robot tests
env -u WAYLAND_DISPLAY XDG_SESSION_TYPE=x11 xvfb-run -a -s "-screen 0 1920x1080x24" robot tests

-a picks a free display number, so several runs can use Xvfb at the same time.

Screen size

Without -s, xvfb-run starts a screen of only 640×480, which is too small for useful screenshots. The display profiles use Full HD.

VS Code does not fill the screen by itself; its window keeps its default size. With the setting window.newWindowDimensions set to maximized and a window manager, it fills the screen, and screenshots and videos show the whole screen size. The example project opens VS Code this way. Without a window manager, Set Viewport Size sets the page to the wanted size instead, as in Videos and slow motion.

Downloads on CI

Open VS Code and Get Electron Executable download VS Code and Electron on first use and cache them in the user’s cache directory. To keep the downloads between CI runs, cache that directory, or give the keywords a cache_dir that the CI caches. Downloads and cache shows how to pass it as a variable.