Files
lit.ruv.wtf/website/data/blog/hidden/moonlight-config.md

14 KiB
Raw Blame History

title, date, author
title date author
moonlight-config 2026-08-29 Max Litruv Boonzaayer

Headless Linux, Sunshine, and Arbitrary Resolution

How I got a headless NVIDIA box running Sunshine to switch to basically whatever resolution Moonlight asks for — including stupid shit like 5120×1440@100, 4096×2160@100, and 2416×1080@60.


The goal

The setup is:

  • RTX 3090
  • no physical monitor
  • Fedora
  • KDE Plasma Wayland
  • Sunshine
  • Moonlight
  • virtual DisplayPort exposed as DP-1

Different clients want different resolutions:

  • Steam Deck — 1280×800@90
  • Steam Link — 1920×1080@60
  • 4K display — 4096×2160@100
  • ultrawide — 5120×1440@100
  • phone — 2416×1080@60

I wanted the host to match whatever the client asks for when it connects, then restore the previous mode when the stream ends.

No manually fucking around with xrandr, no changing KDE settings every time, and no getting stuck at whatever resolution the virtual display happened to boot with.

This is what ended up working on Fedora + KDE Plasma Wayland + NVIDIA 610.x, using a virtual DP connector called DP-1 with an EDID named ModeSwitch.


Why this is such a pain in the arse

There's no actual monitor

The GPU still wants a connected display for modesetting, and Sunshine needs something it can capture through KMS.

So DP-1 is a virtual display with a fake EDID. As far as the GPU is concerned, there's a monitor sitting there.

There isn't.

It's lying.

NVIDIA doesn't like KDE's custom modes

KDE/KScreen will happily create custom resolutions using CVT timings. They'll show up in Display Settings and kscreen-doctor.

Then NVIDIA goes:

nah cunt

and rejects the configuration.

Modes advertised by the EDID as proper native timings work instead:

  • standard detailed timings
  • DisplayID Type I timings

So arbitrary resolutions are handled by changing the EDID rather than creating KDE custom modes.

Sunshine prep commands don't expand $(VAR)

Sunshine exposes the client resolution through:

SUNSHINE_CLIENT_WIDTH
SUNSHINE_CLIENT_HEIGHT
SUNSHINE_CLIENT_FPS

But $(SUNSHINE_CLIENT_WIDTH) in an apps.json prep command doesn't get expanded.

The prep command therefore just calls a wrapper, which reads the variables from its environment.


How it works

  1. Moonlight connects
    • Sunshine prep-cmd
      • sunshine-custom-res apply
        • try existing native EDID mode
        • if it doesn't exist
          • generate+inject EDID
          • hotplug DP-1
          • poke KWin
          • apply new mode
  2. On Disconnect
    • sunshine-custom res undo

The virtual display gets a baseline EDID at boot. Runtime switching replaces that EDID with one containing the exact client mode.

Sunshine captures DP-1 through KMS/NVENC.


Boot-time virtual display

The kernel command line is:

drm.edid_firmware=DP-1:edid/custom-hires.bin video=DP-1:D
^ loads the EDID                             ^ forces connector on boot

The EDID comes from:

/lib/firmware/edid/custom-hires.bin

This gives me a stable DP-1 before KDE or Sunshine starts.

The boot EDID just needs to provide a usable baseline. Runtime switching replaces it when a client connects.


Generating the client EDID

The generator is:

~/.local/bin/gen-sunshine-edid.py

For example:

gen-sunshine-edid.py 5120 1440 100 \
    -o ~/.config/sunshine/client.edid

It builds an NVIDIA-friendly EDID containing the requested mode:

  1. Width is aligned to 8 pixels and height is even
  2. Requested mode goes into DTD #1
  3. A DisplayID Type I block contains the requested mode
  4. A few fallback timings are included
  5. The template's CTA-861 block is omitted

The CTA block is intentionally removed because it can advertise its own preferred VICs and cause KWin to pick the wrong mode.

The generated EDID therefore makes the requested resolution the preferred/native mode instead of relying on KDE's custom mode support.


Injecting the EDID

Handled by:

~/.local/bin/sunshine-vdisplay-inject

The process is:

  1. Write the EDID to DRM debugfs edid_override
  2. Turn DP-1 off
  3. Wait ~1.5 seconds
  4. Turn it back on
  5. Send a change uevent
  6. Tell KWin to reconfigure

The KWin refresh is important because the kernel and KScreen don't necessarily update at the same time.

The kernel can already show the new mode:

cat /sys/class/drm/card0-DP-1/modes

while KScreen is still holding the old mode list.

kscreen-doctor -o

After the connector hotplug and:

org.kde.KWin reconfigure

KScreen reloads the display information and the new native mode becomes available.


Matching the display

The main script is:

~/.local/bin/sunshine-match-display

It supports:

sunshine-match-display do
sunshine-match-display undo

On do it:

  1. Saves the current mode and geometry
  2. Reads SUNSHINE_CLIENT_WIDTH
  3. Reads SUNSHINE_CLIENT_HEIGHT
  4. Reads SUNSHINE_CLIENT_FPS
  5. Looks for an exact native mode
  6. Generates and injects an EDID if necessary
  7. Waits for KScreen to see the new mode
  8. Applies it
  9. Verifies the resulting geometry

undo restores the saved mode.

There is intentionally no nearest-resolution fallback.

If I ask for 2560×1440 and it can't apply 2560×1440, I want it to fail rather than silently leave the display at 4096×2160.

Fuck that.


Sunshine configuration

The global prep command is:

~/.config/sunshine/sunshine.conf
global_prep_cmd = [{
  "do": "/home/litruv/.local/bin/sunshine-match-display do",
  "undo": "/home/litruv/.local/bin/sunshine-match-display undo"
}]

Capture:

capture = kms
output_name = DP-1
encoder = nvenc
minimum_fps_target = 100

KMS grabs the virtual DP directly, avoiding the Wayland screencast path.


Custom resolution apps

Fixed-resolution Moonlight apps can call the same wrapper with explicit values:

{
  "name": "Custom Res 4096x2160@100",
  "image-path": "/home/litruv/.config/sunshine/custom-res-4096x2160-100.png",
  "exclude-global-prep-cmd": true,
  "prep-cmd": [{
    "do": "/home/litruv/.local/bin/sunshine-custom-res apply 4096 2160 100",
    "undo": "/home/litruv/.local/bin/sunshine-custom-res undo"
  }]
}

For automatic client matching:

"do": "/home/litruv/.local/bin/sunshine-custom-res apply"

The wrapper uses the SUNSHINE_CLIENT_* environment variables in that case.

exclude-global-prep-cmd prevents a fixed-resolution app from running the global matcher as well.


The wrapper

sunshine-custom-res is just the entry point Sunshine uses.

sunshine-custom-res apply

Uses the resolution supplied by Sunshine.

sunshine-custom-res apply 5120 1440 100

Uses an explicit resolution.

sunshine-custom-res undo

Restores the previous mode.

The actual display handling stays in sunshine-match-display.


Keeping Sunshine alive

The rest of the setup is mostly there to make the box behave like an always-on streaming server.

Systemd

Sunshine runs as:

app-dev.lizardbyte.app.Sunshine.service

with a sunshine.service symlink.

Drop-ins:

Drop-in Job
override.conf Wait for Wayland/X11 before Sunshine starts
restart.conf Restart Sunshine if it dies
latency.conf Give Sunshine more CPU/IO priority

sunshine-headless-layout.service runs when the graphical session starts and makes sure the virtual DP is active while the physical outputs are off.

Staying awake

sunshine-stay-awake.service

uses systemd-inhibit to prevent sleep.

sunshine-power-inhibit.service

does the same through KDE's PowerDevil D-Bus interface.

Because apparently Linux needs multiple independent systems to decide whether my streaming box is allowed to fucking sleep.

Watchdog

sunshine-watchdog.timer

runs every two minutes and checks port 47989.

If the display exists but Sunshine isn't listening, it restarts the service.

Enable it with:

systemctl --user enable --now sunshine-watchdog.timer

Sunshine performance

Current relevant settings:

capture = kms
output_name = DP-1
encoder = nvenc
minimum_fps_target = 100
nvenc_twopass = quarter_res
nvenc_latency_over_power = enabled
qp = 8

KMS + NVENC is the setup I'm using for high-refresh LAN streaming.


Current client setup

Client Resolution App
Steam Deck OLED 1280×800@90 Custom Res 1280x800@90
Steam Link 1920×1080@60 Custom Res 1920x1080@60
4K display 4096×2160@100 Custom Res 4096x2160@100
Super ultrawide 5120×1440@100 Custom Res 5120x1440@100
Phone 2416×1080@60 Custom Res 2416x1080@60

Debugging

When something breaks, check these in order.

Sunshine

~/.config/sunshine/sunshine.log

Check that the prep command actually ran.

If the automatic command shows:

apply

with no resolution values, the Sunshine environment variables aren't reaching the wrapper.

Match log

~/.config/sunshine/match-display.log

Look for:

request WxH@FPS

and whether it found a native mode or performed an EDID injection.

Kernel EDID

edid-decode /sys/class/drm/card0-DP-1/edid

Check that the preferred timing matches what you requested.

Kernel modes

cat /sys/class/drm/card0-DP-1/modes | head

If the requested mode isn't here, the problem is still on the EDID/kernel side.

KScreen

kscreen-doctor -o

If the kernel has the new mode but KScreen doesn't, the KWin reconfigure/hotplug step isn't doing its job.

Native vs custom

If the mode only appears under Custom modes in kscreen-doctor, NVIDIA will probably reject it.

Use the EDID path instead.

Manual test

You can bypass Sunshine:

SUNSHINE_CLIENT_WIDTH=2560 \
SUNSHINE_CLIENT_HEIGHT=1440 \
SUNSHINE_CLIENT_FPS=100 \
sunshine-custom-res apply

Then:

kscreen-doctor -o | grep Geometry

A few gotchas

EDID injection needs sudo

The debugfs override requires root.

The injector is allowed through sudo with NOPASSWD so Sunshine can do it unattended.

NVIDIA + Wayland

This setup is specifically targeting NVIDIA + KDE Wayland.

Don't assume the same behaviour on AMD or X11.

Resolution changes take a few seconds

An EDID fallback has to:

inject
→ hotplug
→ wait
→ KWin reconfigure
→ KScreen reload
→ apply mode

So there's a brief black screen when switching to a mode that isn't already available.

Bandwidth

4096×2160@100 is roughly 28 Gbps of raw RGB8 data.

DP 1.4 HBR3 can handle it, assuming the rest of the chain isn't the weak link.

Don't use KDE custom resolutions

For this virtual DP, don't use KDE's Add Custom Resolution stuff.

If NVIDIA is going to reject the CVT mode anyway, there's not much point asking KDE to make it.


Files

Display / resolution

File Purpose
~/.local/bin/gen-sunshine-edid.py Generates client EDIDs
~/.local/bin/sunshine-vdisplay-inject Injects EDID and refreshes KWin
~/.local/bin/sunshine-vdisplay-up Forces the boot EDID
~/.local/bin/sunshine-match-display Switches/restores display modes
~/.local/bin/sunshine-custom-res Sunshine prep wrapper
~/.local/bin/sunshine-headless-layout Sets up the virtual DP layout
~/.local/bin/gen-sunshine-custom-res-icons.py Generates app icons
~/.config/sunshine/client.edid Last generated EDID
~/.config/sunshine/match-display.log Resolution switch log
~/.config/sunshine/last-display-mode.env Saved mode for undo
~/.config/sunshine/apps.json Moonlight app definitions
~/.config/sunshine/custom-res-*.png Custom resolution icons
~/.config/sunshine/sunshine.conf Sunshine config
/lib/firmware/edid/custom-hires.bin Boot EDID
~/edid-custom-hires.bin Source copy of boot EDID

Sunshine / systemd

File Purpose
~/.local/bin/import-gui-env.sh Waits for the compositor
~/.config/systemd/user/import-gui-env.service Starts the environment setup
~/.config/systemd/user/sunshine-headless-layout.service Sets up the headless layout
~/.config/systemd/user/sunshine-stay-awake.service Prevents sleep
~/.config/systemd/user/sunshine-power-inhibit.service KDE power inhibition
~/.config/systemd/user/sunshine-watchdog.{service,timer} Sunshine watchdog
~/.config/systemd/user/app-dev.lizardbyte.app.Sunshine.service.d/ Sunshine drop-ins
~/.local/bin/sunshine-renice Sunshine priority
~/.local/bin/sunshine-watchdog Watchdog script

TL;DR

The trick is stop trying to make KDE create arbitrary NVIDIA modes.

Give the GPU a virtual DP with an EDID, then change that EDID when a client asks for a mode KDE doesn't already have:

  1. Boot with a baseline EDID
  2. Read the client's resolution from Sunshine
  3. Generate an EDID containing that exact mode
  4. Inject it into DP-1
  5. Hotplug the connector and reconfigure KWin
  6. Apply the native mode with kscreen-doctor
  7. Capture with KMS + NVENC
  8. Restore the previous mode on disconnect

Once that's in place, arbitrary Moonlight resolutions are basically just an EDID + shell script problem.

Which is considerably less painful than trying to convince KDE and NVIDIA to cooperate.