跪拜 Guibai
← Back to the summary

Starling Turns Flutter Into a Full Linux Desktop Environment

This uncanny sense of déjà vu — yet another project combining Swift and the Flutter Engine. But Starling goes further: after rewriting and porting Flutter's Dart Framework directly into Swift, while keeping Flutter Engine's rendering engine, it actually ended up building a full Linux desktop environment.

That's right, he used Flutter to write an independent Linux desktop UI system. Starling is akin to using Flutter's Widget, Element, RenderObject, Layer, and GPU rendering system to rebuild a complete desktop environment similar to GNOME, KDE, or COSMIC.

Because it takes over DRM/KMS, displays, mouse and keyboard, Wayland clients, X11 clients, window management, Dock, Spaces, Mission Control, and desktop Portals.

Additionally, the window surfaces of external applications are wrapped by Starling as Flutter TextureWidgets, entering the same Flutter Layer Tree along with title bars, shadows, rounded corners, blur, and animations.

Absolutely mind-blowing, isn't it? I never expected Flutter to finally reach the point of a complete desktop. Even Canonical only uses Flutter for system apps, while this project directly built a new desktop environment.

Currently, the repository is mainly divided into two parts. starling-build/starling is the main repository for the Swift Framework, desktop Shell, and applications:

Directory Purpose
sdk/ Port of the Flutter Dart Framework to Swift
shell/ Desktop Shell, Window Manager, Wayland Compositor, X11 Server
apps/ Settings, File Manager, Terminal, Calculator, App Store, etc.
registry/ Application registry and store catalog
build/ Session, Deb packages, run and install tools
docs/ Build, installation, porting, and design documentation

The sdk/ here is a complete Swift Framework port implementation of Flutter, without the Dart VM. shell/ mainly contains the compositor, window manager, Dock, Spaces, and Portal desktop support.

Then there is starling-build/starling-engine, a fork of the Flutter Engine, mainly adding two parts:

Flutter Engine
├── linux_drm/
│   └── DRM/KMS, GBM/EGL, libinput, libseat, multi-display Embedder
│
└── lib/ui/swift/
    └── Flutter Engine ↔ Swift Framework bridging layer

This repository is a fork based on Flutter 3.38.6, with Starling's increments involving roughly 184 files.

Starling already supports installation on Ubuntu 26.04 and can be switched to a real desktop session from the GDM login screen, similar to:

gdm3
  ↓
gdm-wayland-session
  ↓
/usr/libexec/starling-session
  ↓
DesktopShellApp --drm

The entire project has achieved almost complete capability support, such as:

This means Starling itself is a Wayland Compositor and X Server. The entire framework can be roughly viewed as:

                         Linux User Space
┌───────────────────────────────────────────────────────────┐
│                    DesktopShellApp                         │
│                         Swift                             │
│                                                           │
│  ┌────────────────── FlutterSwift Framework ───────────┐  │
│  │ Widgets → Elements → RenderObjects → Layers         │  │
│  │ Gestures / Animation / Scheduler / Semantics        │  │
│  │ Fluent UI / Macos UI / Shell UI                     │  │
│  └─────────────────────────┬────────────────────────────┘  │
│                            │ Scene                         │
│  ┌──────────────┐    ┌─────▼──────────┐    ┌────────────┐  │
│  │ Wayland      │    │ External       │    │ X11 Server │  │
│  │ Server       │───▶│ Texture        │◀──│ DRI3/      │  │
│  │ xdg-shell    │    │ Registry       │    │ Present    │  │
│  └──────────────┘    └─────┬──────────┘    └────────────┘  │
│         dma-buf / wl_shm    │ EGLImage / GL Texture        │
└─────────────────────────────┼─────────────────────────────┘
                              │
                    Flutter Engine C/C++
              Scene → Rasterizer → GPU Compositor
                              │
                         GBM / EGL
                              │
                          DRM / KMS
                              │
                       Display CRTC

A very crucial inversion can be seen here, because the ordinary Flutter architecture is:

Flutter App → Wayland/X11 → Desktop Compositor → Display

But now Starling's implementation is:

Wayland/X11 App → Starling → Flutter Layer Tree → Display

Here, Flutter is no longer an App UI framework; it has directly become the server-side and final compositing layer of the desktop system. Starling calls this client-to-server inversion.

So Starling essentially replaced Dart with Swift, and it didn't transform it into a SwiftUI style; instead, it retained Flutter's original model. For example, the Swift version of Widget is still a configuration object for Element:

open class Widget {
    let key: Key?

    func createElement() -> Element

    static func canUpdate(
        _ oldWidget: Widget,
        _ newWidget: Widget
    ) -> Bool
}

The update rule is still runtimeType && key. The foundation is still StatelessWidget creating StatelessElement, and StatefulWidget creating StatefulElement. Even the core of the Swift version of setState hasn't changed:

func setState(_ mutation: () -> Void) {
    mutation()
    element.markNeedsBuild()
}

Even the multi-child update algorithm was ported. So it seems the author simply doesn't like Dart. However, there are some troublesome parts, such as how to map Dart mixins to Swift.

This is one of the most troublesome parts of porting the Flutter Framework, because Flutter heavily relies on Dart mixins. The current approach is:

So scenarios like these can only be imitated through mapping. Anyway, the author would rather rewrite in Swift than use Dart.

Next is the core of Starling, such as "How does the desktop display directly on the screen?". In Starling, it starts with DesktopShellApp --drm. The entire startup process is similar to:

runApp(
    FluentApp(
        home: DesktopShell()
    )
)

var callbacks = createRuntimeCallbacks()

let drmView = fl_drm_view_create(
    assetsPath,
    icuPath,
    &callbacks
)

let engine = fl_drm_view_get_engine(drmView)

setupWayland(engine)
setupX11(engine)
setupTextures(engine)

fl_drm_view_run(drmView)

Of course, this is simplified code. The actual entry point first builds the Widget Tree, then initializes the display, EGL, and Flutter Engine through fl_drm_view_create. fl_drm_view_create is responsible for:

Open /dev/dri/card*
    ↓
Request DRM Master / libseat session
    ↓
Enumerate Connector / CRTC / Plane
    ↓
Select display mode
    ↓
Create GBM Surface
    ↓
Create EGL Display / Context
    ↓
Initialize Flutter Engine
    ↓
Start libinput

For this part, its public C API only exposes one fl_drm_view.h to Swift, which reduces the coupling between the Swift Shell and the Engine Fork to some extent. The Shell only depends on the stable C API.

Then comes the most interesting part of Starling. Under the Starling desktop, the windows of ordinary Linux applications like Chrome, VS Code, and GIMP are also placed, clipped, and animated by Starling as Flutter Widgets.

Of course, what becomes a Widget here is not the Chrome program itself, but only the image output by the Chrome window. Chrome is still an independent process, internally still using Chromium, Blink, and its own GPU rendering pipeline. Starling just takes the final rendered window image from Chrome and wraps it into a TextureWidget. You can understand it as:

Chrome is responsible for drawing web content
        ↓
Obtains a constantly updating GPU image
        ↓
Starling puts this image into a TextureWidget
        ↓
Flutter then handles window position, rounded corners, shadows, scaling, and animations

Similar to a video player in Flutter, Flutter is not responsible for decoding the video, only for displaying each frame of texture provided by the player.

What Starling does to the Chrome window is actually very similar to how Flutter displays video textures: taking the window image continuously submitted by a real Linux application and placing it as a continuously updating external Texture into the Widget Tree.

Because in Wayland, Chrome doesn't draw directly onto the monitor. It first completes web page rendering in its own GPU Buffer, for example:

Chrome GPU Process
    ↓
Draws tab bar, web page, text, and images
    ↓
Gets a complete window Buffer

Then Chrome tells the Wayland compositor "Here is my newly drawn window image, please display it". Protocol-wise, this roughly corresponds to:

wl_surface.attach(buffer); // Attach the new image to the window;
wl_surface.damage(...); // Tell the compositor which areas have changed;
wl_surface.commit(); // Officially submit this frame.

When an ordinary GNOME desktop receives this Buffer, it hands it over to Mutter for compositing. When Starling receives it, it hands this Buffer to the Flutter Engine.

However, Starling doesn't copy Chrome's image into a picture. When Starling supports linux-dmabuf, it receives a DMA-BUF file descriptor. You can understand DMA-BUF as:

A cache handle that can be shared between different processes and different GPU components.

So what Chrome gives Starling is the handle and format information of the GPU Buffer. The relevant information includes:

fd          File descriptor of the shared Buffer
width       Width
height      Height
stride      Bytes per row
fourcc      Pixel format, e.g., XRGB8888
modifier    Memory layout of the GPU Buffer

Then Starling registers this GPU Buffer as a Flutter Texture, thus entering the familiar Flutter rendering pipeline.

If DMA-BUF is not supported, Starling currently has roughly four scenarios:

Wayland + linux-dmabuf
→ GPU Buffer imported directly, mainly a zero-copy path

Wayland + wl_shm
→ Application provides CPU shared memory, Starling uploads it as a GPU Texture

X11 + DRI3/Present
→ Obtain DMA-BUF, take the GPU import path

X11 + PutImage/ShmPutImage
→ Obtain CPU pixels, then upload to GPU

So the ideal path is GPU → GPU, while the compatibility path is CPU pixels → GPU Texture. Normally, Chrome, Electron, GTK4, Qt6 usually take the DMA-BUF path, unless some older apps take the CPU upload path.

Moreover, since the Chrome window content has become a TextureWidget, you can even wrap it with a series of things like title bars, shadows, rounded corners, resize handles, and even Flutter animations. The final window image output by Chrome becomes a layer in the Flutter scene tree that can be freely transformed.

Finally, Starling has a Wayland Server implemented in several thousand lines of C, and then Swift's WaylandIntegration is responsible for converting protocol events into window system behaviors, mainly supporting:

That is, it does not use wlroots, but instead implements its own protocols within the project.

And the project also has a built-in minimal X11 Server, listening on :1, supporting DRI3/Present GPU Buffer, and PutImage/ShmPutImage software paths. When an X11 window is mapped, it also registers an External Texture, and upon receiving PresentPixmap, imports the DMA-BUF into the Texture Registry.

So X11 is not through XWayland either; it's closer to a built-in X Server oriented towards Starling's compositing model.

It's truly a heavy undertaking, and he has even completed multi-window and multi-display support. The traditional approach is for the desktop compositor to implement its own Scene Graph, and then the application Framework to implement another set. Starling's idea is indeed audacious:

Since Flutter already has a complete Scene Graph, Layout, Animation, and GPU Compositor, why can't external windows also become Widgets?

He has taken Flutter to a new level, right? Not embedding a UI Framework in the Compositor, but directly making the UI Framework become the Compositor.

Currently, the project has 335,000 lines of Swift/C/C++ code, of which about 273,000 lines belong to the Flutter to Swift framework port, and the desktop Shell, Wayland/X11 server, and applications are about 62,000 lines. This is basically the largest Flutter community open-source project I have ever seen, and also the most audacious, bar none.

Links

https://starling.build/

https://github.com/starling-build/starling

Comments

Top 1 from juejin.cn, machine-translated. The original thread is authoritative.

伯伯熊勞

I don't really get it, but it sounds impressive.