Claiming Full Pipewire integration. I need to add a button to the UI to allow screen sharing
This commit is contained in:
ganome
parent
0f8c3dd0b1
commit
a92a5887d7
GPG Key ID:
944DE53336D81B83
193
PROGRESS-SCREENCAPTURE.md
Normal file
193
PROGRESS-SCREENCAPTURE.md
Normal file
@ -0,0 +1,193 @@
|
||||
# Screen Capture Implementation Progress
|
||||
|
||||
## Overview
|
||||
Implementation of screen capture functionality for SCAR Chat with support for multiple backends:
|
||||
- **Wayland/Hyprland**: xdg-desktop-portal + Pipewire
|
||||
- **X11**: FFmpeg with x11grab
|
||||
- **Windows**: FFmpeg with GDI
|
||||
|
||||
## Current Status: 🟡 In Progress
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Backend Detection
|
||||
- ✅ Auto-detection via environment variables (WAYLAND_DISPLAY, HYPRLAND_INSTANCE_SIGNATURE)
|
||||
- ✅ Fallback mechanism (Portal → X11 → Windows)
|
||||
- ✅ Manual backend selection support
|
||||
|
||||
### Wayland/Hyprland Implementation (Priority)
|
||||
**Status**: 🟡 In Progress - **CORRECTED TO USE HYPRLAND PORTAL**
|
||||
|
||||
**Critical Architecture Understanding**:
|
||||
- xdg-desktop-portal-hyprland **implements** org.freedesktop.portal.ScreenCast (standard API)
|
||||
- We use **sdbus-c++** library (NOT libdbus-1) to communicate with the portal
|
||||
- The portal handles Hyprland-specific details internally (via hyprland-share-picker)
|
||||
- From client perspective: call standard portal API → portal shows hyprland-share-picker → get Pipewire stream
|
||||
|
||||
**Dependencies**:
|
||||
- **sdbus-c++** (required for DBus communication) ✅ Installed
|
||||
- **libpipewire-0.3** (for stream handling) ✅ Installed
|
||||
- **xdg-desktop-portal-hyprland** (runtime requirement - provides hyprland-share-picker)
|
||||
- **spa-utils** (for spa_hook structure)
|
||||
|
||||
**Implementation Tasks**:
|
||||
- [x] Switch from libdbus-1 to sdbus-c++
|
||||
- [x] Use standard org.freedesktop.portal.Desktop.ScreenCast interface
|
||||
- [x] Update CMakeLists.txt with pkg-config for sdbus-c++ and pipewire
|
||||
- [x] Screen capture class structure with backend selection
|
||||
- [x] initPortalConnection() - sdbus session bus connection
|
||||
- [x] cleanupPortalConnection() - Resource cleanup
|
||||
- [x] createPortalSession() - Use sdbus-c++ for CreateSession method with unique handles
|
||||
- [x] selectPortalSources() - Use sdbus-c++ for SelectSources (portal shows hyprland-share-picker)
|
||||
- [x] startPortalSession() - Use sdbus-c++ for Start method
|
||||
- [x] openPipeWireRemote() - Get file descriptor from portal using UnixFd
|
||||
- [x] getStreamsNodeId() - Query session Streams property to get actual node_id
|
||||
- [x] initPipewire() - Complete implementation with stream connection, listeners, and thread loop
|
||||
- [x] onStreamProcess() - Frame callback implementation that dequeues buffers and invokes user callback
|
||||
- [x] onStreamParamChanged() - Handle resolution/format changes and update dimensions
|
||||
- [x] cleanupPipewire() - Stop thread loop and cleanup resources properly
|
||||
- [ ] Test end-to-end screen capture flow
|
||||
- [ ] Frame buffer memory management optimization
|
||||
- [ ] Error handling and session recovery
|
||||
- [ ] Restore token support for session persistence
|
||||
|
||||
**Notes**:
|
||||
- Standard portal API is service name: `org.freedesktop.portal.Desktop`
|
||||
- Object path: `/org/freedesktop/portal/desktop`
|
||||
- Interface: `org.freedesktop.portal.ScreenCast`
|
||||
- When SelectSources is called, xdph automatically launches hyprland-share-picker GUI
|
||||
- User selection is handled transparently - we just get back session handle + Pipewire node
|
||||
|
||||
**Technical Details**:
|
||||
```cpp
|
||||
// XDG Desktop Portal ScreenCast API Workflow:
|
||||
// 1. org.freedesktop.portal.ScreenCast.CreateSession(options) -> session_handle
|
||||
// - Creates session object for this screen cast
|
||||
//
|
||||
// 2. org.freedesktop.portal.ScreenCast.SelectSources(session_handle, options)
|
||||
// - options.types: MONITOR(1), WINDOW(2), VIRTUAL(4)
|
||||
// - options.multiple: allow selecting multiple sources
|
||||
// - options.cursor_mode: Hidden(1), Embedded(2), Metadata(4)
|
||||
// - options.persist_mode: DoNotPersist(0), WhileRunning(1), UntilRevoked(2)
|
||||
//
|
||||
// 3. org.freedesktop.portal.ScreenCast.Start(session_handle, parent_window, options)
|
||||
// - User selects screen/window via portal UI
|
||||
// - Response includes: streams array with [(node_id, properties)]
|
||||
// - Each stream has: id, position, size, source_type
|
||||
// - Returns restore_token for future sessions
|
||||
//
|
||||
// 4. org.freedesktop.portal.ScreenCast.OpenPipeWireRemote(session_handle) -> fd
|
||||
// - Returns file descriptor for PipeWire connection
|
||||
//
|
||||
// 5. Pipewire Connection:
|
||||
// - pw_context_connect_fd(fd) creates pw_core
|
||||
// - pw_stream_new() with node_id from Step 3
|
||||
// - pw_stream_add_listener() for frame callbacks
|
||||
// - pw_stream_connect() to start streaming
|
||||
//
|
||||
// 6. Frame Processing:
|
||||
// - on_process() callback receives spa_buffer with frame data
|
||||
// - Extract video/raw format (RGB, YUV, etc.)
|
||||
// - Invoke FrameCallback with decoded data
|
||||
```
|
||||
|
||||
### X11 Implementation (Fallback)
|
||||
**Status**: 🔴 Not Started
|
||||
|
||||
**Dependencies**:
|
||||
- FFmpeg (libavformat, libavcodec, libavutil, libavdevice)
|
||||
- X11 libraries
|
||||
|
||||
**Implementation Tasks**:
|
||||
- [ ] FFmpeg context initialization
|
||||
- [ ] x11grab input device configuration
|
||||
- [ ] Frame extraction and decoding
|
||||
- [ ] Frame callback integration
|
||||
- [ ] Display selection (multi-monitor support)
|
||||
|
||||
### Windows Implementation (Future)
|
||||
**Status**: 🔴 Not Started
|
||||
|
||||
**Dependencies**:
|
||||
- FFmpeg with GDI support
|
||||
|
||||
**Implementation Tasks**:
|
||||
- [ ] FFmpeg GDI grabber setup
|
||||
- [ ] Frame processing pipeline
|
||||
- [ ] Display enumeration
|
||||
|
||||
---
|
||||
|
||||
## Testing Plan
|
||||
|
||||
### Unit Tests
|
||||
- [ ] Backend detection on different environments
|
||||
- [ ] Frame callback invocation
|
||||
- [ ] Start/stop lifecycle
|
||||
- [ ] Memory leak verification
|
||||
|
||||
### Integration Tests
|
||||
- [ ] Wayland/Hyprland capture on real desktop
|
||||
- [ ] X11 capture verification
|
||||
- [ ] Multi-monitor scenarios
|
||||
- [ ] Permission denial handling
|
||||
|
||||
### Performance Tests
|
||||
- [ ] Frame rate consistency (target: 30 FPS)
|
||||
- [ ] CPU usage profiling
|
||||
- [ ] Memory usage under continuous capture
|
||||
|
||||
---
|
||||
|
||||
## Known Issues & Limitations
|
||||
|
||||
### Current
|
||||
- All backends are stubs (no actual implementation)
|
||||
- No frame encoding/compression
|
||||
- No multi-monitor selection UI
|
||||
|
||||
### Future Considerations
|
||||
- Portal permissions may require user interaction each session
|
||||
- Hyprland-specific optimizations possible via hyprland-share-picker
|
||||
- Frame rate limiting needed to prevent CPU overload
|
||||
- Consider hardware encoding for lower CPU usage
|
||||
|
||||
---
|
||||
|
||||
## Code Locations
|
||||
- **Header**: `client/media/screen_capture.h`
|
||||
- **Implementation**: `client/media/screen_capture.cpp`
|
||||
- **Dependencies**: `CMakeLists.txt` (client section)
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
1. Implement Pipewire + Portal screen capture for Wayland/Hyprland
|
||||
2. Test on Hyprland environment
|
||||
3. Implement X11 fallback
|
||||
4. Add frame encoding for network transmission
|
||||
5. Integrate with video streaming protocol
|
||||
|
||||
---
|
||||
|
||||
## Session Log
|
||||
|
||||
### Session 1 - December 7, 2025
|
||||
- **Completed**: Authentication system fully working (plaintext → salt → argon2 verification)
|
||||
- **Fixed**: Message deserialization bug (async buffer capture issue in both client and server)
|
||||
- **Status**: Ready to begin screen capture implementation
|
||||
- **Decision**: Prioritize Wayland/Hyprland implementation due to target environment
|
||||
|
||||
### Session 2 - December 7, 2025 (Current)
|
||||
- **Researched**: xdg-desktop-portal-hyprland specifications and org.freedesktop.portal.ScreenCast API
|
||||
- **Implemented**:
|
||||
- Screen capture header with forward declarations for DBus/Pipewire types
|
||||
- Basic structure with backend detection and selection
|
||||
- DBus initialization and cleanup functions
|
||||
- Pipewire initialization skeleton (loop, context creation)
|
||||
- Platform-specific compilation (#ifdef __linux__)
|
||||
- startPortalCapture() workflow outline (6-step process)
|
||||
- **TODO**: Implement actual DBus method calls for portal communication
|
||||
- **Next**: Implement createPortalSession() with proper DBus message building
|
||||
@ -40,8 +40,14 @@ target_link_libraries(scarchat PRIVATE
|
||||
|
||||
# Platform-specific media libraries
|
||||
if(UNIX AND NOT APPLE)
|
||||
# Find sdbus-c++ for xdg-desktop-portal communication
|
||||
find_package(PkgConfig REQUIRED)
|
||||
pkg_check_modules(SDBUS REQUIRED IMPORTED_TARGET sdbus-c++)
|
||||
pkg_check_modules(PIPEWIRE REQUIRED IMPORTED_TARGET libpipewire-0.3)
|
||||
|
||||
target_link_libraries(scarchat PRIVATE
|
||||
pipewire-0.3
|
||||
PkgConfig::SDBUS
|
||||
PkgConfig::PIPEWIRE
|
||||
avcodec
|
||||
avformat
|
||||
avutil
|
||||
|
||||
@ -1,25 +1,42 @@
|
||||
#include "screen_capture.h"
|
||||
#include <iostream>
|
||||
#include <cstring>
|
||||
#include <random>
|
||||
#include <sstream>
|
||||
#include <iomanip>
|
||||
#include <unistd.h>
|
||||
|
||||
// TODO: Include FFmpeg headers when implementing
|
||||
// extern "C" {
|
||||
// #include <libavcodec/avcodec.h>
|
||||
// #include <libavformat/avformat.h>
|
||||
// #include <libavutil/avutil.h>
|
||||
// }
|
||||
|
||||
// TODO: Include DBus/Portal headers for Wayland
|
||||
// #include <dbus/dbus.h>
|
||||
#ifdef __linux__
|
||||
#include <sdbus-c++/sdbus-c++.h>
|
||||
#include <pipewire/pipewire.h>
|
||||
#include <spa/param/video/format-utils.h>
|
||||
#include <spa/param/video/type-info.h>
|
||||
#include <spa/param/format-utils.h>
|
||||
#include <spa/param/video/format.h>
|
||||
#include <spa/utils/hook.h>
|
||||
#include <spa/pod/builder.h>
|
||||
#include <QImage>
|
||||
#endif
|
||||
|
||||
namespace scar {
|
||||
|
||||
ScreenCapture::ScreenCapture()
|
||||
: capturing_(false) {
|
||||
: capturing_(false),
|
||||
pw_loop_(nullptr),
|
||||
pw_stream_(nullptr),
|
||||
pw_context_(nullptr),
|
||||
frame_width_(0),
|
||||
frame_height_(0) {
|
||||
backend_ = detectBestBackend();
|
||||
#ifdef __linux__
|
||||
memset(&stream_listener_, 0, sizeof(stream_listener_));
|
||||
#endif
|
||||
}
|
||||
|
||||
ScreenCapture::~ScreenCapture() {
|
||||
stop();
|
||||
cleanupPipewire();
|
||||
cleanupPortalConnection();
|
||||
}
|
||||
|
||||
ScreenCaptureBackend ScreenCapture::detectBestBackend() {
|
||||
@ -53,17 +70,19 @@ bool ScreenCapture::start(ScreenCaptureBackend backend) {
|
||||
backend_ = backend;
|
||||
|
||||
switch (backend_) {
|
||||
case ScreenCaptureBackend::PORTAL_PIPEWIRE:
|
||||
return startPortalCapture();
|
||||
case ScreenCaptureBackend::FFMPEG_X11:
|
||||
return startX11Capture();
|
||||
case ScreenCaptureBackend::FFMPEG_WAYLAND:
|
||||
case ScreenCaptureBackend::PORTAL_PIPEWIRE:
|
||||
return startWaylandCapture();
|
||||
case ScreenCaptureBackend::FFMPEG_WINDOWS:
|
||||
return startWindowsCapture();
|
||||
}
|
||||
|
||||
default:
|
||||
std::cerr << "Unknown backend" << std::endl;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
void ScreenCapture::stop() {
|
||||
if (!capturing_) {
|
||||
@ -72,10 +91,8 @@ void ScreenCapture::stop() {
|
||||
|
||||
std::cout << "Stopping screen capture..." << std::endl;
|
||||
|
||||
// TODO: Clean up FFmpeg resources
|
||||
// avformat_close_input(&formatCtx_);
|
||||
// avcodec_free_context(&codecCtx_);
|
||||
// av_frame_free(&frame_);
|
||||
cleanupPipewire();
|
||||
cleanupPortalConnection();
|
||||
|
||||
capturing_ = false;
|
||||
}
|
||||
@ -85,53 +102,518 @@ void ScreenCapture::setFrameCallback(FrameCallback callback) {
|
||||
}
|
||||
|
||||
bool ScreenCapture::startX11Capture() {
|
||||
std::cout << "Starting X11 screen capture via FFmpeg..." << std::endl;
|
||||
|
||||
// TODO: Implement X11 capture using FFmpeg
|
||||
// 1. Open x11grab input device
|
||||
// 2. Set up video codec (H.264 or raw)
|
||||
// 3. Read frames in loop
|
||||
// 4. Call frameCallback_ for each frame
|
||||
// Example: ffmpeg -f x11grab -i :0.0 -c:v libx264 ...
|
||||
|
||||
capturing_ = true;
|
||||
return true;
|
||||
std::cout << "X11 screen capture not yet implemented" << std::endl;
|
||||
// TODO: Implement FFmpeg x11grab
|
||||
return false;
|
||||
}
|
||||
|
||||
bool ScreenCapture::startWaylandCapture() {
|
||||
std::cout << "Starting Wayland screen capture via xdg-desktop-portal + Pipewire..." << std::endl;
|
||||
|
||||
// TODO: Implement portal-based capture
|
||||
// 1. Connect to DBus
|
||||
// 2. Call org.freedesktop.portal.ScreenCast.CreateSession
|
||||
// 3. Select sources via SelectSources
|
||||
// 4. Start the session
|
||||
// 5. Retrieve Pipewire node ID
|
||||
// 6. Connect to Pipewire stream using node ID
|
||||
// 7. Read frames and call frameCallback_
|
||||
|
||||
// Note: May need to prompt user for screen selection dialog
|
||||
|
||||
capturing_ = true;
|
||||
return true;
|
||||
std::cout << "FFmpeg Wayland capture not yet implemented" << std::endl;
|
||||
// TODO: Implement FFmpeg Wayland
|
||||
return false;
|
||||
}
|
||||
|
||||
bool ScreenCapture::startWindowsCapture() {
|
||||
std::cout << "Starting Windows screen capture via FFmpeg..." << std::endl;
|
||||
|
||||
// TODO: Implement Windows GDI capture using FFmpeg
|
||||
// 1. Open gdigrab input device
|
||||
// 2. Set up video codec
|
||||
// 3. Read frames in loop
|
||||
// 4. Call frameCallback_ for each frame
|
||||
// Example: ffmpeg -f gdigrab -i desktop -c:v libx264 ...
|
||||
|
||||
capturing_ = true;
|
||||
return true;
|
||||
std::cout << "Windows screen capture not yet implemented" << std::endl;
|
||||
// TODO: Implement FFmpeg GDI
|
||||
return false;
|
||||
}
|
||||
|
||||
bool ScreenCapture::startPortalCapture() {
|
||||
return startWaylandCapture(); // Same implementation
|
||||
#ifndef __linux__
|
||||
std::cerr << "Portal capture only supported on Linux" << std::endl;
|
||||
return false;
|
||||
#else
|
||||
std::cout << "Starting Portal + Pipewire screen capture..." << std::endl;
|
||||
|
||||
// Step 1: Initialize portal connection
|
||||
if (!initPortalConnection()) {
|
||||
std::cerr << "Failed to connect to xdg-desktop-portal" << std::endl;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Step 2: Create portal session
|
||||
std::string session_handle = createPortalSession();
|
||||
if (session_handle.empty()) {
|
||||
std::cerr << "Failed to create portal session" << std::endl;
|
||||
cleanupPortalConnection();
|
||||
return false;
|
||||
}
|
||||
|
||||
// Step 3: Select sources (monitors/windows)
|
||||
if (!selectPortalSources(session_handle)) {
|
||||
std::cerr << "Failed to select portal sources" << std::endl;
|
||||
cleanupPortalConnection();
|
||||
return false;
|
||||
}
|
||||
|
||||
// Step 4: Start session and get Pipewire node ID
|
||||
uint32_t node_id = 0;
|
||||
if (!startPortalSession(session_handle, node_id)) {
|
||||
std::cerr << "Failed to start portal session" << std::endl;
|
||||
cleanupPortalConnection();
|
||||
return false;
|
||||
}
|
||||
|
||||
std::cout << "Got Pipewire node ID: " << node_id << std::endl;
|
||||
|
||||
// Step 5: Open Pipewire remote
|
||||
int pw_fd = openPipeWireRemote(session_handle);
|
||||
if (pw_fd < 0) {
|
||||
std::cerr << "Failed to open Pipewire remote" << std::endl;
|
||||
cleanupPortalConnection();
|
||||
return false;
|
||||
}
|
||||
|
||||
// Step 6: Initialize Pipewire and connect to stream
|
||||
if (!initPipewire(pw_fd, node_id)) {
|
||||
std::cerr << "Failed to initialize Pipewire" << std::endl;
|
||||
cleanupPortalConnection();
|
||||
return false;
|
||||
}
|
||||
|
||||
capturing_ = true;
|
||||
std::cout << "Screen capture started successfully" << std::endl;
|
||||
return true;
|
||||
#endif
|
||||
}
|
||||
|
||||
#ifdef __linux__
|
||||
bool ScreenCapture::initPortalConnection() {
|
||||
try {
|
||||
// Create session bus connection
|
||||
portal_connection_ = sdbus::createSessionBusConnection();
|
||||
|
||||
// Create proxy for ScreenCast portal interface
|
||||
// Using standard org.freedesktop.portal.Desktop interface
|
||||
// xdg-desktop-portal-hyprland implements this standard interface
|
||||
screencast_proxy_ = sdbus::createProxy(
|
||||
*portal_connection_,
|
||||
sdbus::ServiceName{"org.freedesktop.portal.Desktop"},
|
||||
sdbus::ObjectPath{"/org/freedesktop/portal/desktop"}
|
||||
);
|
||||
|
||||
std::cout << "Connected to xdg-desktop-portal (Hyprland implementation)" << std::endl;
|
||||
return true;
|
||||
}
|
||||
catch (const sdbus::Error& e) {
|
||||
std::cerr << "Failed to connect to portal: " << e.what() << std::endl;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
void ScreenCapture::cleanupPortalConnection() {
|
||||
screencast_proxy_.reset();
|
||||
portal_connection_.reset();
|
||||
}
|
||||
|
||||
std::string ScreenCapture::createPortalSession() {
|
||||
try {
|
||||
// Generate unique handle for this request
|
||||
std::random_device rd;
|
||||
std::mt19937 gen(rd());
|
||||
std::uniform_int_distribution<> dis(0, 999999);
|
||||
|
||||
std::stringstream ss;
|
||||
ss << "/org/freedesktop/portal/desktop/request/"
|
||||
<< getpid() << "_" << dis(gen);
|
||||
std::string request_handle = ss.str();
|
||||
|
||||
ss.str("");
|
||||
ss << "/org/freedesktop/portal/desktop/session/"
|
||||
<< getpid() << "_" << dis(gen);
|
||||
std::string session_handle = ss.str();
|
||||
|
||||
std::cout << "Creating portal session..." << std::endl;
|
||||
std::cout << " Request handle: " << request_handle << std::endl;
|
||||
std::cout << " Session handle: " << session_handle << std::endl;
|
||||
|
||||
// Build options dictionary
|
||||
std::map<std::string, sdbus::Variant> options;
|
||||
options["handle_token"] = sdbus::Variant{request_handle.substr(request_handle.rfind('/') + 1)};
|
||||
options["session_handle_token"] = sdbus::Variant{session_handle.substr(session_handle.rfind('/') + 1)};
|
||||
|
||||
// Call CreateSession
|
||||
sdbus::ObjectPath response_path;
|
||||
screencast_proxy_->callMethod("CreateSession")
|
||||
.onInterface("org.freedesktop.portal.ScreenCast")
|
||||
.withArguments(options)
|
||||
.storeResultsTo(response_path);
|
||||
|
||||
std::cout << "Session created: " << session_handle << std::endl;
|
||||
session_path_ = session_handle;
|
||||
return session_handle;
|
||||
}
|
||||
catch (const sdbus::Error& e) {
|
||||
std::cerr << "Failed to create portal session: " << e.what() << std::endl;
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
bool ScreenCapture::selectPortalSources(const std::string& session_handle) {
|
||||
try {
|
||||
// Generate unique request handle
|
||||
std::random_device rd;
|
||||
std::mt19937 gen(rd());
|
||||
std::uniform_int_distribution<> dis(0, 999999);
|
||||
|
||||
std::stringstream ss;
|
||||
ss << "/org/freedesktop/portal/desktop/request/"
|
||||
<< getpid() << "_" << dis(gen);
|
||||
std::string request_handle = ss.str();
|
||||
|
||||
std::cout << "Selecting portal sources..." << std::endl;
|
||||
|
||||
// Build options dictionary
|
||||
std::map<std::string, sdbus::Variant> options;
|
||||
options["handle_token"] = sdbus::Variant{request_handle.substr(request_handle.rfind('/') + 1)};
|
||||
options["types"] = sdbus::Variant{uint32_t(1 | 2)}; // MONITOR (1) | WINDOW (2)
|
||||
options["multiple"] = sdbus::Variant{false};
|
||||
options["cursor_mode"] = sdbus::Variant{uint32_t(2)}; // Embedded cursor
|
||||
|
||||
// Call SelectSources
|
||||
sdbus::ObjectPath response_path;
|
||||
screencast_proxy_->callMethod("SelectSources")
|
||||
.onInterface("org.freedesktop.portal.ScreenCast")
|
||||
.withArguments(sdbus::ObjectPath{session_handle}, options)
|
||||
.storeResultsTo(response_path);
|
||||
|
||||
std::cout << "Sources selected (user will see hyprland-share-picker)" << std::endl;
|
||||
return true;
|
||||
}
|
||||
catch (const sdbus::Error& e) {
|
||||
std::cerr << "Failed to select portal sources: " << e.what() << std::endl;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
bool ScreenCapture::startPortalSession(const std::string& session_handle, uint32_t& node_id) {
|
||||
try {
|
||||
// Generate unique request handle
|
||||
std::random_device rd;
|
||||
std::mt19937 gen(rd());
|
||||
std::uniform_int_distribution<> dis(0, 999999);
|
||||
|
||||
std::stringstream ss;
|
||||
ss << "/org/freedesktop/portal/desktop/request/"
|
||||
<< getpid() << "_" << dis(gen);
|
||||
std::string request_handle = ss.str();
|
||||
|
||||
std::cout << "Starting portal session..." << std::endl;
|
||||
|
||||
// Build options dictionary
|
||||
std::map<std::string, sdbus::Variant> options;
|
||||
options["handle_token"] = sdbus::Variant{request_handle.substr(request_handle.rfind('/') + 1)};
|
||||
|
||||
// Call Start
|
||||
sdbus::ObjectPath response_path;
|
||||
screencast_proxy_->callMethod("Start")
|
||||
.onInterface("org.freedesktop.portal.ScreenCast")
|
||||
.withArguments(sdbus::ObjectPath{session_handle}, std::string(""), options)
|
||||
.storeResultsTo(response_path);
|
||||
|
||||
std::cout << "Portal session started" << std::endl;
|
||||
|
||||
// Get the actual node_id from the session's Streams property
|
||||
node_id = getStreamsNodeId(session_handle);
|
||||
|
||||
if (node_id == 0) {
|
||||
std::cerr << "Failed to get valid node_id from streams" << std::endl;
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
catch (const sdbus::Error& e) {
|
||||
std::cerr << "Failed to start portal session: " << e.what() << std::endl;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
int ScreenCapture::openPipeWireRemote(const std::string& session_handle) {
|
||||
try {
|
||||
std::cout << "Opening PipeWire remote..." << std::endl;
|
||||
|
||||
// Build options dictionary (empty for now)
|
||||
std::map<std::string, sdbus::Variant> options;
|
||||
|
||||
// Call OpenPipeWireRemote
|
||||
sdbus::UnixFd pipewire_fd;
|
||||
screencast_proxy_->callMethod("OpenPipeWireRemote")
|
||||
.onInterface("org.freedesktop.portal.ScreenCast")
|
||||
.withArguments(sdbus::ObjectPath{session_handle}, options)
|
||||
.storeResultsTo(pipewire_fd);
|
||||
|
||||
// Extract the actual file descriptor from UnixFd
|
||||
int fd = dup(pipewire_fd.get());
|
||||
|
||||
if (fd < 0) {
|
||||
std::cerr << "Failed to duplicate PipeWire file descriptor" << std::endl;
|
||||
return -1;
|
||||
}
|
||||
|
||||
std::cout << "PipeWire remote opened, fd: " << fd << std::endl;
|
||||
return fd;
|
||||
}
|
||||
catch (const sdbus::Error& e) {
|
||||
std::cerr << "Failed to open PipeWire remote: " << e.what() << std::endl;
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
uint32_t ScreenCapture::getStreamsNodeId(const std::string& session_handle) {
|
||||
try {
|
||||
std::cout << "Getting streams from session..." << std::endl;
|
||||
|
||||
// Create a proxy for the session object
|
||||
auto session_proxy = sdbus::createProxy(
|
||||
*portal_connection_,
|
||||
sdbus::ServiceName{"org.freedesktop.portal.Desktop"},
|
||||
sdbus::ObjectPath{session_handle}
|
||||
);
|
||||
|
||||
// Get the Streams property from org.freedesktop.portal.Session interface
|
||||
auto streams_variant = session_proxy->getProperty("Streams")
|
||||
.onInterface("org.freedesktop.portal.Session");
|
||||
|
||||
// Streams is a(ua{sv}) - array of (node_id, properties)
|
||||
auto streams = streams_variant.get<std::vector<sdbus::Struct<uint32_t, std::map<std::string, sdbus::Variant>>>>();
|
||||
|
||||
if (streams.empty()) {
|
||||
std::cerr << "No streams available in session" << std::endl;
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Get the first stream's node_id
|
||||
uint32_t node_id = std::get<0>(streams[0]);
|
||||
auto& properties = std::get<1>(streams[0]);
|
||||
|
||||
std::cout << "Found stream node_id: " << node_id << std::endl;
|
||||
|
||||
// Print stream properties if available
|
||||
if (properties.count("size")) {
|
||||
auto size = properties["size"].get<sdbus::Struct<int32_t, int32_t>>();
|
||||
std::cout << " Stream size: " << std::get<0>(size) << "x" << std::get<1>(size) << std::endl;
|
||||
}
|
||||
|
||||
return node_id;
|
||||
}
|
||||
catch (const sdbus::Error& e) {
|
||||
std::cerr << "Failed to get streams: " << e.what() << std::endl;
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
bool ScreenCapture::initPipewire(int fd, uint32_t node_id) {
|
||||
pw_init(nullptr, nullptr);
|
||||
|
||||
std::cout << "Initializing PipeWire with fd: " << fd << ", node_id: " << node_id << std::endl;
|
||||
|
||||
pw_loop_ = pw_thread_loop_new("screen-capture", nullptr);
|
||||
if (!pw_loop_) {
|
||||
std::cerr << "Failed to create Pipewire loop" << std::endl;
|
||||
return false;
|
||||
}
|
||||
|
||||
pw_context_ = pw_context_new(
|
||||
pw_thread_loop_get_loop(pw_loop_),
|
||||
nullptr, 0
|
||||
);
|
||||
if (!pw_context_) {
|
||||
std::cerr << "Failed to create Pipewire context" << std::endl;
|
||||
pw_thread_loop_destroy(pw_loop_);
|
||||
pw_loop_ = nullptr;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Connect to the PipeWire remote using the file descriptor
|
||||
pw_core* core = pw_context_connect_fd(
|
||||
pw_context_,
|
||||
fd,
|
||||
nullptr, 0
|
||||
);
|
||||
if (!core) {
|
||||
std::cerr << "Failed to connect to PipeWire remote" << std::endl;
|
||||
cleanupPipewire();
|
||||
return false;
|
||||
}
|
||||
|
||||
// Create stream properties
|
||||
pw_properties* props = pw_properties_new(
|
||||
PW_KEY_MEDIA_TYPE, "Video",
|
||||
PW_KEY_MEDIA_CATEGORY, "Capture",
|
||||
PW_KEY_MEDIA_ROLE, "Screen",
|
||||
nullptr
|
||||
);
|
||||
|
||||
// Create the stream
|
||||
pw_stream_ = pw_stream_new(
|
||||
core,
|
||||
"screen-capture-stream",
|
||||
props
|
||||
);
|
||||
|
||||
if (!pw_stream_) {
|
||||
std::cerr << "Failed to create PipeWire stream" << std::endl;
|
||||
pw_core_disconnect(core);
|
||||
cleanupPipewire();
|
||||
return false;
|
||||
}
|
||||
|
||||
// Set up stream events
|
||||
static const pw_stream_events stream_events = {
|
||||
PW_VERSION_STREAM_EVENTS,
|
||||
nullptr, // destroy
|
||||
nullptr, // state_changed
|
||||
nullptr, // control_info
|
||||
nullptr, // io_changed
|
||||
onStreamParamChanged, // param_changed
|
||||
nullptr, // add_buffer
|
||||
nullptr, // remove_buffer
|
||||
onStreamProcess, // process
|
||||
nullptr, // drained
|
||||
nullptr, // command
|
||||
nullptr, // trigger_done
|
||||
};
|
||||
|
||||
pw_stream_add_listener(
|
||||
pw_stream_,
|
||||
&stream_listener_,
|
||||
&stream_events,
|
||||
this // userdata pointer
|
||||
);
|
||||
|
||||
// Connect to the node
|
||||
uint8_t buffer[1024];
|
||||
spa_pod_builder pod_builder = SPA_POD_BUILDER_INIT(buffer, sizeof(buffer));
|
||||
|
||||
// Request any video format (we'll adapt to what we get)
|
||||
const struct spa_pod* params[1];
|
||||
params[0] = reinterpret_cast<const struct spa_pod*>(spa_pod_builder_add_object(
|
||||
&pod_builder,
|
||||
SPA_TYPE_OBJECT_Format, SPA_PARAM_EnumFormat,
|
||||
SPA_FORMAT_mediaType, SPA_POD_Id(SPA_MEDIA_TYPE_video),
|
||||
SPA_FORMAT_mediaSubtype, SPA_POD_Id(SPA_MEDIA_SUBTYPE_raw)
|
||||
));
|
||||
|
||||
int result = pw_stream_connect(
|
||||
pw_stream_,
|
||||
PW_DIRECTION_INPUT,
|
||||
node_id,
|
||||
(pw_stream_flags)(PW_STREAM_FLAG_AUTOCONNECT | PW_STREAM_FLAG_MAP_BUFFERS),
|
||||
params, 1
|
||||
);
|
||||
|
||||
if (result < 0) {
|
||||
std::cerr << "Failed to connect stream to node " << node_id << ": " << strerror(-result) << std::endl;
|
||||
pw_core_disconnect(core);
|
||||
cleanupPipewire();
|
||||
return false;
|
||||
}
|
||||
|
||||
// Start the thread loop
|
||||
if (pw_thread_loop_start(pw_loop_) < 0) {
|
||||
std::cerr << "Failed to start PipeWire thread loop" << std::endl;
|
||||
pw_core_disconnect(core);
|
||||
cleanupPipewire();
|
||||
return false;
|
||||
}
|
||||
|
||||
std::cout << "PipeWire stream connected and running" << std::endl;
|
||||
return true;
|
||||
}
|
||||
|
||||
void ScreenCapture::cleanupPipewire() {
|
||||
if (pw_loop_) {
|
||||
pw_thread_loop_stop(pw_loop_);
|
||||
}
|
||||
|
||||
if (pw_stream_) {
|
||||
pw_stream_destroy(pw_stream_);
|
||||
pw_stream_ = nullptr;
|
||||
}
|
||||
|
||||
if (pw_context_) {
|
||||
pw_context_destroy(pw_context_);
|
||||
pw_context_ = nullptr;
|
||||
}
|
||||
|
||||
if (pw_loop_) {
|
||||
pw_thread_loop_destroy(pw_loop_);
|
||||
pw_loop_ = nullptr;
|
||||
}
|
||||
|
||||
pw_deinit();
|
||||
}
|
||||
|
||||
void ScreenCapture::onStreamProcess(void* userdata) {
|
||||
auto* self = static_cast<ScreenCapture*>(userdata);
|
||||
|
||||
pw_buffer* buffer = pw_stream_dequeue_buffer(self->pw_stream_);
|
||||
if (!buffer) {
|
||||
std::cerr << "No buffer available" << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
spa_buffer* spa_buf = buffer->buffer;
|
||||
if (!spa_buf || !spa_buf->datas || !spa_buf->datas[0].data) {
|
||||
pw_stream_queue_buffer(self->pw_stream_, buffer);
|
||||
return;
|
||||
}
|
||||
|
||||
// Get frame data
|
||||
spa_data& data = spa_buf->datas[0];
|
||||
uint8_t* frame_data = static_cast<uint8_t*>(data.data);
|
||||
uint32_t size = data.chunk->size;
|
||||
|
||||
// Invoke the callback if set
|
||||
if (self->frameCallback_) {
|
||||
// Convert to vector for callback
|
||||
std::vector<uint8_t> frame_vec(frame_data, frame_data + size);
|
||||
self->frameCallback_(frame_vec, self->frame_width_, self->frame_height_);
|
||||
}
|
||||
|
||||
// Return buffer to stream
|
||||
pw_stream_queue_buffer(self->pw_stream_, buffer);
|
||||
}
|
||||
|
||||
void ScreenCapture::onStreamParamChanged(void* userdata, uint32_t id, const struct spa_pod* param) {
|
||||
auto* self = static_cast<ScreenCapture*>(userdata);
|
||||
|
||||
if (!param || id != SPA_PARAM_Format) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Parse video format
|
||||
spa_video_info_raw video_info;
|
||||
if (spa_format_video_raw_parse(param, &video_info) < 0) {
|
||||
std::cerr << "Failed to parse video format" << std::endl;
|
||||
return;
|
||||
}
|
||||
|
||||
// Update our stored dimensions
|
||||
self->frame_width_ = video_info.size.width;
|
||||
self->frame_height_ = video_info.size.height;
|
||||
|
||||
std::cout << "Stream format changed:" << std::endl;
|
||||
std::cout << " Resolution: " << self->frame_width_ << "x" << self->frame_height_ << std::endl;
|
||||
std::cout << " Format: " << video_info.format << std::endl;
|
||||
std::cout << " Framerate: " << video_info.framerate.num << "/" << video_info.framerate.denom << std::endl;
|
||||
}
|
||||
|
||||
#else
|
||||
// Stub implementations for non-Linux platforms
|
||||
bool ScreenCapture::initPortalConnection() { return false; }
|
||||
void ScreenCapture::cleanupPortalConnection() {}
|
||||
std::string ScreenCapture::createPortalSession() { return ""; }
|
||||
bool ScreenCapture::selectPortalSources(const std::string&) { return false; }
|
||||
bool ScreenCapture::startPortalSession(const std::string&, uint32_t&) { return false; }
|
||||
int ScreenCapture::openPipeWireRemote(const std::string&) { return -1; }
|
||||
uint32_t ScreenCapture::getStreamsNodeId(const std::string&) { return 0; }
|
||||
bool ScreenCapture::initPipewire(int, uint32_t) { return false; }
|
||||
void ScreenCapture::cleanupPipewire() {}
|
||||
void ScreenCapture::onStreamProcess(void*) {}
|
||||
void ScreenCapture::onStreamParamChanged(void*, uint32_t, const struct spa_pod*) {}
|
||||
#endif
|
||||
|
||||
} // namespace scar
|
||||
|
||||
@ -5,6 +5,21 @@
|
||||
#include <vector>
|
||||
#include <string>
|
||||
|
||||
#ifdef __linux__
|
||||
#include <spa/utils/hook.h>
|
||||
#include <spa/pod/pod.h>
|
||||
#endif
|
||||
|
||||
// Forward declarations for Pipewire/sdbus to avoid header pollution
|
||||
struct pw_thread_loop;
|
||||
struct pw_stream;
|
||||
struct pw_context;
|
||||
|
||||
namespace sdbus {
|
||||
class IConnection;
|
||||
class IProxy;
|
||||
}
|
||||
|
||||
namespace scar {
|
||||
|
||||
enum class ScreenCaptureBackend {
|
||||
@ -44,18 +59,43 @@ private:
|
||||
bool startWindowsCapture();
|
||||
bool startPortalCapture();
|
||||
|
||||
// sdbus Portal methods
|
||||
bool initPortalConnection();
|
||||
void cleanupPortalConnection();
|
||||
std::string createPortalSession();
|
||||
bool selectPortalSources(const std::string& session_handle);
|
||||
bool startPortalSession(const std::string& session_handle, uint32_t& node_id);
|
||||
int openPipeWireRemote(const std::string& session_handle);
|
||||
uint32_t getStreamsNodeId(const std::string& session_handle);
|
||||
|
||||
// Pipewire methods
|
||||
bool initPipewire(int fd, uint32_t node_id);
|
||||
void cleanupPipewire();
|
||||
static void onStreamProcess(void* userdata);
|
||||
static void onStreamParamChanged(void* userdata, uint32_t id, const struct spa_pod* param);
|
||||
|
||||
bool capturing_;
|
||||
ScreenCaptureBackend backend_;
|
||||
FrameCallback frameCallback_;
|
||||
|
||||
// sdbus members
|
||||
std::shared_ptr<sdbus::IConnection> portal_connection_;
|
||||
std::unique_ptr<sdbus::IProxy> screencast_proxy_;
|
||||
std::string session_path_;
|
||||
|
||||
// Pipewire members
|
||||
pw_thread_loop* pw_loop_;
|
||||
pw_stream* pw_stream_;
|
||||
pw_context* pw_context_;
|
||||
spa_hook stream_listener_;
|
||||
|
||||
int frame_width_;
|
||||
int frame_height_;
|
||||
|
||||
// TODO: FFmpeg-specific members
|
||||
// AVFormatContext* formatCtx_;
|
||||
// AVCodecContext* codecCtx_;
|
||||
// AVFrame* frame_;
|
||||
|
||||
// TODO: DBus/Portal-specific members for Wayland
|
||||
// DBusConnection* dbusConn_;
|
||||
// pw_stream* pipewireStream_;
|
||||
};
|
||||
|
||||
} // namespace scar
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user