diff --git a/PROGRESS-SCREENCAPTURE.md b/PROGRESS-SCREENCAPTURE.md new file mode 100644 index 0000000..f89f549 --- /dev/null +++ b/PROGRESS-SCREENCAPTURE.md @@ -0,0 +1,193 @@ +# Screen Capture Implementation Progress + +## Overview +Implementation of screen capture functionality for SCAR Chat with support for multiple backends: +- **Wayland/Hyprland**: xdg-desktop-portal + Pipewire +- **X11**: FFmpeg with x11grab +- **Windows**: FFmpeg with GDI + +## Current Status: 🟡 In Progress + +--- + +## Architecture + +### Backend Detection +- ✅ Auto-detection via environment variables (WAYLAND_DISPLAY, HYPRLAND_INSTANCE_SIGNATURE) +- ✅ Fallback mechanism (Portal → X11 → Windows) +- ✅ Manual backend selection support + +### Wayland/Hyprland Implementation (Priority) +**Status**: 🟡 In Progress - **CORRECTED TO USE HYPRLAND PORTAL** + +**Critical Architecture Understanding**: +- xdg-desktop-portal-hyprland **implements** org.freedesktop.portal.ScreenCast (standard API) +- We use **sdbus-c++** library (NOT libdbus-1) to communicate with the portal +- The portal handles Hyprland-specific details internally (via hyprland-share-picker) +- From client perspective: call standard portal API → portal shows hyprland-share-picker → get Pipewire stream + +**Dependencies**: +- **sdbus-c++** (required for DBus communication) ✅ Installed +- **libpipewire-0.3** (for stream handling) ✅ Installed +- **xdg-desktop-portal-hyprland** (runtime requirement - provides hyprland-share-picker) +- **spa-utils** (for spa_hook structure) + +**Implementation Tasks**: +- [x] Switch from libdbus-1 to sdbus-c++ +- [x] Use standard org.freedesktop.portal.Desktop.ScreenCast interface +- [x] Update CMakeLists.txt with pkg-config for sdbus-c++ and pipewire +- [x] Screen capture class structure with backend selection +- [x] initPortalConnection() - sdbus session bus connection +- [x] cleanupPortalConnection() - Resource cleanup +- [x] createPortalSession() - Use sdbus-c++ for CreateSession method with unique handles +- [x] selectPortalSources() - Use sdbus-c++ for SelectSources (portal shows hyprland-share-picker) +- [x] startPortalSession() - Use sdbus-c++ for Start method +- [x] openPipeWireRemote() - Get file descriptor from portal using UnixFd +- [x] getStreamsNodeId() - Query session Streams property to get actual node_id +- [x] initPipewire() - Complete implementation with stream connection, listeners, and thread loop +- [x] onStreamProcess() - Frame callback implementation that dequeues buffers and invokes user callback +- [x] onStreamParamChanged() - Handle resolution/format changes and update dimensions +- [x] cleanupPipewire() - Stop thread loop and cleanup resources properly +- [ ] Test end-to-end screen capture flow +- [ ] Frame buffer memory management optimization +- [ ] Error handling and session recovery +- [ ] Restore token support for session persistence + +**Notes**: +- Standard portal API is service name: `org.freedesktop.portal.Desktop` +- Object path: `/org/freedesktop/portal/desktop` +- Interface: `org.freedesktop.portal.ScreenCast` +- When SelectSources is called, xdph automatically launches hyprland-share-picker GUI +- User selection is handled transparently - we just get back session handle + Pipewire node + +**Technical Details**: +```cpp +// XDG Desktop Portal ScreenCast API Workflow: +// 1. org.freedesktop.portal.ScreenCast.CreateSession(options) -> session_handle +// - Creates session object for this screen cast +// +// 2. org.freedesktop.portal.ScreenCast.SelectSources(session_handle, options) +// - options.types: MONITOR(1), WINDOW(2), VIRTUAL(4) +// - options.multiple: allow selecting multiple sources +// - options.cursor_mode: Hidden(1), Embedded(2), Metadata(4) +// - options.persist_mode: DoNotPersist(0), WhileRunning(1), UntilRevoked(2) +// +// 3. org.freedesktop.portal.ScreenCast.Start(session_handle, parent_window, options) +// - User selects screen/window via portal UI +// - Response includes: streams array with [(node_id, properties)] +// - Each stream has: id, position, size, source_type +// - Returns restore_token for future sessions +// +// 4. org.freedesktop.portal.ScreenCast.OpenPipeWireRemote(session_handle) -> fd +// - Returns file descriptor for PipeWire connection +// +// 5. Pipewire Connection: +// - pw_context_connect_fd(fd) creates pw_core +// - pw_stream_new() with node_id from Step 3 +// - pw_stream_add_listener() for frame callbacks +// - pw_stream_connect() to start streaming +// +// 6. Frame Processing: +// - on_process() callback receives spa_buffer with frame data +// - Extract video/raw format (RGB, YUV, etc.) +// - Invoke FrameCallback with decoded data +``` + +### X11 Implementation (Fallback) +**Status**: 🔴 Not Started + +**Dependencies**: +- FFmpeg (libavformat, libavcodec, libavutil, libavdevice) +- X11 libraries + +**Implementation Tasks**: +- [ ] FFmpeg context initialization +- [ ] x11grab input device configuration +- [ ] Frame extraction and decoding +- [ ] Frame callback integration +- [ ] Display selection (multi-monitor support) + +### Windows Implementation (Future) +**Status**: 🔴 Not Started + +**Dependencies**: +- FFmpeg with GDI support + +**Implementation Tasks**: +- [ ] FFmpeg GDI grabber setup +- [ ] Frame processing pipeline +- [ ] Display enumeration + +--- + +## Testing Plan + +### Unit Tests +- [ ] Backend detection on different environments +- [ ] Frame callback invocation +- [ ] Start/stop lifecycle +- [ ] Memory leak verification + +### Integration Tests +- [ ] Wayland/Hyprland capture on real desktop +- [ ] X11 capture verification +- [ ] Multi-monitor scenarios +- [ ] Permission denial handling + +### Performance Tests +- [ ] Frame rate consistency (target: 30 FPS) +- [ ] CPU usage profiling +- [ ] Memory usage under continuous capture + +--- + +## Known Issues & Limitations + +### Current +- All backends are stubs (no actual implementation) +- No frame encoding/compression +- No multi-monitor selection UI + +### Future Considerations +- Portal permissions may require user interaction each session +- Hyprland-specific optimizations possible via hyprland-share-picker +- Frame rate limiting needed to prevent CPU overload +- Consider hardware encoding for lower CPU usage + +--- + +## Code Locations +- **Header**: `client/media/screen_capture.h` +- **Implementation**: `client/media/screen_capture.cpp` +- **Dependencies**: `CMakeLists.txt` (client section) + +--- + +## Next Steps +1. Implement Pipewire + Portal screen capture for Wayland/Hyprland +2. Test on Hyprland environment +3. Implement X11 fallback +4. Add frame encoding for network transmission +5. Integrate with video streaming protocol + +--- + +## Session Log + +### Session 1 - December 7, 2025 +- **Completed**: Authentication system fully working (plaintext → salt → argon2 verification) +- **Fixed**: Message deserialization bug (async buffer capture issue in both client and server) +- **Status**: Ready to begin screen capture implementation +- **Decision**: Prioritize Wayland/Hyprland implementation due to target environment + +### Session 2 - December 7, 2025 (Current) +- **Researched**: xdg-desktop-portal-hyprland specifications and org.freedesktop.portal.ScreenCast API +- **Implemented**: + - Screen capture header with forward declarations for DBus/Pipewire types + - Basic structure with backend detection and selection + - DBus initialization and cleanup functions + - Pipewire initialization skeleton (loop, context creation) + - Platform-specific compilation (#ifdef __linux__) + - startPortalCapture() workflow outline (6-step process) +- **TODO**: Implement actual DBus method calls for portal communication +- **Next**: Implement createPortalSession() with proper DBus message building diff --git a/client/CMakeLists.txt b/client/CMakeLists.txt index 925c6e6..971cd17 100644 --- a/client/CMakeLists.txt +++ b/client/CMakeLists.txt @@ -40,8 +40,14 @@ target_link_libraries(scarchat PRIVATE # Platform-specific media libraries if(UNIX AND NOT APPLE) + # Find sdbus-c++ for xdg-desktop-portal communication + find_package(PkgConfig REQUIRED) + pkg_check_modules(SDBUS REQUIRED IMPORTED_TARGET sdbus-c++) + pkg_check_modules(PIPEWIRE REQUIRED IMPORTED_TARGET libpipewire-0.3) + target_link_libraries(scarchat PRIVATE - pipewire-0.3 + PkgConfig::SDBUS + PkgConfig::PIPEWIRE avcodec avformat avutil diff --git a/client/media/screen_capture.cpp b/client/media/screen_capture.cpp index 495d12b..a943a70 100644 --- a/client/media/screen_capture.cpp +++ b/client/media/screen_capture.cpp @@ -1,25 +1,42 @@ #include "screen_capture.h" #include +#include +#include +#include +#include +#include -// TODO: Include FFmpeg headers when implementing -// extern "C" { -// #include -// #include -// #include -// } - -// TODO: Include DBus/Portal headers for Wayland -// #include +#ifdef __linux__ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#endif namespace scar { ScreenCapture::ScreenCapture() - : capturing_(false) { + : capturing_(false), + pw_loop_(nullptr), + pw_stream_(nullptr), + pw_context_(nullptr), + frame_width_(0), + frame_height_(0) { backend_ = detectBestBackend(); +#ifdef __linux__ + memset(&stream_listener_, 0, sizeof(stream_listener_)); +#endif } ScreenCapture::~ScreenCapture() { stop(); + cleanupPipewire(); + cleanupPortalConnection(); } ScreenCaptureBackend ScreenCapture::detectBestBackend() { @@ -53,16 +70,18 @@ bool ScreenCapture::start(ScreenCaptureBackend backend) { backend_ = backend; switch (backend_) { + case ScreenCaptureBackend::PORTAL_PIPEWIRE: + return startPortalCapture(); case ScreenCaptureBackend::FFMPEG_X11: return startX11Capture(); case ScreenCaptureBackend::FFMPEG_WAYLAND: - case ScreenCaptureBackend::PORTAL_PIPEWIRE: return startWaylandCapture(); case ScreenCaptureBackend::FFMPEG_WINDOWS: return startWindowsCapture(); + default: + std::cerr << "Unknown backend" << std::endl; + return false; } - - return false; } void ScreenCapture::stop() { @@ -72,10 +91,8 @@ void ScreenCapture::stop() { std::cout << "Stopping screen capture..." << std::endl; - // TODO: Clean up FFmpeg resources - // avformat_close_input(&formatCtx_); - // avcodec_free_context(&codecCtx_); - // av_frame_free(&frame_); + cleanupPipewire(); + cleanupPortalConnection(); capturing_ = false; } @@ -85,53 +102,518 @@ void ScreenCapture::setFrameCallback(FrameCallback callback) { } bool ScreenCapture::startX11Capture() { - std::cout << "Starting X11 screen capture via FFmpeg..." << std::endl; - - // TODO: Implement X11 capture using FFmpeg - // 1. Open x11grab input device - // 2. Set up video codec (H.264 or raw) - // 3. Read frames in loop - // 4. Call frameCallback_ for each frame - // Example: ffmpeg -f x11grab -i :0.0 -c:v libx264 ... - - capturing_ = true; - return true; + std::cout << "X11 screen capture not yet implemented" << std::endl; + // TODO: Implement FFmpeg x11grab + return false; } bool ScreenCapture::startWaylandCapture() { - std::cout << "Starting Wayland screen capture via xdg-desktop-portal + Pipewire..." << std::endl; - - // TODO: Implement portal-based capture - // 1. Connect to DBus - // 2. Call org.freedesktop.portal.ScreenCast.CreateSession - // 3. Select sources via SelectSources - // 4. Start the session - // 5. Retrieve Pipewire node ID - // 6. Connect to Pipewire stream using node ID - // 7. Read frames and call frameCallback_ - - // Note: May need to prompt user for screen selection dialog - - capturing_ = true; - return true; + std::cout << "FFmpeg Wayland capture not yet implemented" << std::endl; + // TODO: Implement FFmpeg Wayland + return false; } bool ScreenCapture::startWindowsCapture() { - std::cout << "Starting Windows screen capture via FFmpeg..." << std::endl; - - // TODO: Implement Windows GDI capture using FFmpeg - // 1. Open gdigrab input device - // 2. Set up video codec - // 3. Read frames in loop - // 4. Call frameCallback_ for each frame - // Example: ffmpeg -f gdigrab -i desktop -c:v libx264 ... - - capturing_ = true; - return true; + std::cout << "Windows screen capture not yet implemented" << std::endl; + // TODO: Implement FFmpeg GDI + return false; } bool ScreenCapture::startPortalCapture() { - return startWaylandCapture(); // Same implementation +#ifndef __linux__ + std::cerr << "Portal capture only supported on Linux" << std::endl; + return false; +#else + std::cout << "Starting Portal + Pipewire screen capture..." << std::endl; + + // Step 1: Initialize portal connection + if (!initPortalConnection()) { + std::cerr << "Failed to connect to xdg-desktop-portal" << std::endl; + return false; + } + + // Step 2: Create portal session + std::string session_handle = createPortalSession(); + if (session_handle.empty()) { + std::cerr << "Failed to create portal session" << std::endl; + cleanupPortalConnection(); + return false; + } + + // Step 3: Select sources (monitors/windows) + if (!selectPortalSources(session_handle)) { + std::cerr << "Failed to select portal sources" << std::endl; + cleanupPortalConnection(); + return false; + } + + // Step 4: Start session and get Pipewire node ID + uint32_t node_id = 0; + if (!startPortalSession(session_handle, node_id)) { + std::cerr << "Failed to start portal session" << std::endl; + cleanupPortalConnection(); + return false; + } + + std::cout << "Got Pipewire node ID: " << node_id << std::endl; + + // Step 5: Open Pipewire remote + int pw_fd = openPipeWireRemote(session_handle); + if (pw_fd < 0) { + std::cerr << "Failed to open Pipewire remote" << std::endl; + cleanupPortalConnection(); + return false; + } + + // Step 6: Initialize Pipewire and connect to stream + if (!initPipewire(pw_fd, node_id)) { + std::cerr << "Failed to initialize Pipewire" << std::endl; + cleanupPortalConnection(); + return false; + } + + capturing_ = true; + std::cout << "Screen capture started successfully" << std::endl; + return true; +#endif } +#ifdef __linux__ +bool ScreenCapture::initPortalConnection() { + try { + // Create session bus connection + portal_connection_ = sdbus::createSessionBusConnection(); + + // Create proxy for ScreenCast portal interface + // Using standard org.freedesktop.portal.Desktop interface + // xdg-desktop-portal-hyprland implements this standard interface + screencast_proxy_ = sdbus::createProxy( + *portal_connection_, + sdbus::ServiceName{"org.freedesktop.portal.Desktop"}, + sdbus::ObjectPath{"/org/freedesktop/portal/desktop"} + ); + + std::cout << "Connected to xdg-desktop-portal (Hyprland implementation)" << std::endl; + return true; + } + catch (const sdbus::Error& e) { + std::cerr << "Failed to connect to portal: " << e.what() << std::endl; + return false; + } +} + +void ScreenCapture::cleanupPortalConnection() { + screencast_proxy_.reset(); + portal_connection_.reset(); +} + +std::string ScreenCapture::createPortalSession() { + try { + // Generate unique handle for this request + std::random_device rd; + std::mt19937 gen(rd()); + std::uniform_int_distribution<> dis(0, 999999); + + std::stringstream ss; + ss << "/org/freedesktop/portal/desktop/request/" + << getpid() << "_" << dis(gen); + std::string request_handle = ss.str(); + + ss.str(""); + ss << "/org/freedesktop/portal/desktop/session/" + << getpid() << "_" << dis(gen); + std::string session_handle = ss.str(); + + std::cout << "Creating portal session..." << std::endl; + std::cout << " Request handle: " << request_handle << std::endl; + std::cout << " Session handle: " << session_handle << std::endl; + + // Build options dictionary + std::map options; + options["handle_token"] = sdbus::Variant{request_handle.substr(request_handle.rfind('/') + 1)}; + options["session_handle_token"] = sdbus::Variant{session_handle.substr(session_handle.rfind('/') + 1)}; + + // Call CreateSession + sdbus::ObjectPath response_path; + screencast_proxy_->callMethod("CreateSession") + .onInterface("org.freedesktop.portal.ScreenCast") + .withArguments(options) + .storeResultsTo(response_path); + + std::cout << "Session created: " << session_handle << std::endl; + session_path_ = session_handle; + return session_handle; + } + catch (const sdbus::Error& e) { + std::cerr << "Failed to create portal session: " << e.what() << std::endl; + return ""; + } +} + +bool ScreenCapture::selectPortalSources(const std::string& session_handle) { + try { + // Generate unique request handle + std::random_device rd; + std::mt19937 gen(rd()); + std::uniform_int_distribution<> dis(0, 999999); + + std::stringstream ss; + ss << "/org/freedesktop/portal/desktop/request/" + << getpid() << "_" << dis(gen); + std::string request_handle = ss.str(); + + std::cout << "Selecting portal sources..." << std::endl; + + // Build options dictionary + std::map options; + options["handle_token"] = sdbus::Variant{request_handle.substr(request_handle.rfind('/') + 1)}; + options["types"] = sdbus::Variant{uint32_t(1 | 2)}; // MONITOR (1) | WINDOW (2) + options["multiple"] = sdbus::Variant{false}; + options["cursor_mode"] = sdbus::Variant{uint32_t(2)}; // Embedded cursor + + // Call SelectSources + sdbus::ObjectPath response_path; + screencast_proxy_->callMethod("SelectSources") + .onInterface("org.freedesktop.portal.ScreenCast") + .withArguments(sdbus::ObjectPath{session_handle}, options) + .storeResultsTo(response_path); + + std::cout << "Sources selected (user will see hyprland-share-picker)" << std::endl; + return true; + } + catch (const sdbus::Error& e) { + std::cerr << "Failed to select portal sources: " << e.what() << std::endl; + return false; + } +} + +bool ScreenCapture::startPortalSession(const std::string& session_handle, uint32_t& node_id) { + try { + // Generate unique request handle + std::random_device rd; + std::mt19937 gen(rd()); + std::uniform_int_distribution<> dis(0, 999999); + + std::stringstream ss; + ss << "/org/freedesktop/portal/desktop/request/" + << getpid() << "_" << dis(gen); + std::string request_handle = ss.str(); + + std::cout << "Starting portal session..." << std::endl; + + // Build options dictionary + std::map options; + options["handle_token"] = sdbus::Variant{request_handle.substr(request_handle.rfind('/') + 1)}; + + // Call Start + sdbus::ObjectPath response_path; + screencast_proxy_->callMethod("Start") + .onInterface("org.freedesktop.portal.ScreenCast") + .withArguments(sdbus::ObjectPath{session_handle}, std::string(""), options) + .storeResultsTo(response_path); + + std::cout << "Portal session started" << std::endl; + + // Get the actual node_id from the session's Streams property + node_id = getStreamsNodeId(session_handle); + + if (node_id == 0) { + std::cerr << "Failed to get valid node_id from streams" << std::endl; + return false; + } + + return true; + } + catch (const sdbus::Error& e) { + std::cerr << "Failed to start portal session: " << e.what() << std::endl; + return false; + } +} + +int ScreenCapture::openPipeWireRemote(const std::string& session_handle) { + try { + std::cout << "Opening PipeWire remote..." << std::endl; + + // Build options dictionary (empty for now) + std::map options; + + // Call OpenPipeWireRemote + sdbus::UnixFd pipewire_fd; + screencast_proxy_->callMethod("OpenPipeWireRemote") + .onInterface("org.freedesktop.portal.ScreenCast") + .withArguments(sdbus::ObjectPath{session_handle}, options) + .storeResultsTo(pipewire_fd); + + // Extract the actual file descriptor from UnixFd + int fd = dup(pipewire_fd.get()); + + if (fd < 0) { + std::cerr << "Failed to duplicate PipeWire file descriptor" << std::endl; + return -1; + } + + std::cout << "PipeWire remote opened, fd: " << fd << std::endl; + return fd; + } + catch (const sdbus::Error& e) { + std::cerr << "Failed to open PipeWire remote: " << e.what() << std::endl; + return -1; + } +} + +uint32_t ScreenCapture::getStreamsNodeId(const std::string& session_handle) { + try { + std::cout << "Getting streams from session..." << std::endl; + + // Create a proxy for the session object + auto session_proxy = sdbus::createProxy( + *portal_connection_, + sdbus::ServiceName{"org.freedesktop.portal.Desktop"}, + sdbus::ObjectPath{session_handle} + ); + + // Get the Streams property from org.freedesktop.portal.Session interface + auto streams_variant = session_proxy->getProperty("Streams") + .onInterface("org.freedesktop.portal.Session"); + + // Streams is a(ua{sv}) - array of (node_id, properties) + auto streams = streams_variant.get>>>(); + + if (streams.empty()) { + std::cerr << "No streams available in session" << std::endl; + return 0; + } + + // Get the first stream's node_id + uint32_t node_id = std::get<0>(streams[0]); + auto& properties = std::get<1>(streams[0]); + + std::cout << "Found stream node_id: " << node_id << std::endl; + + // Print stream properties if available + if (properties.count("size")) { + auto size = properties["size"].get>(); + std::cout << " Stream size: " << std::get<0>(size) << "x" << std::get<1>(size) << std::endl; + } + + return node_id; + } + catch (const sdbus::Error& e) { + std::cerr << "Failed to get streams: " << e.what() << std::endl; + return 0; + } +} + +bool ScreenCapture::initPipewire(int fd, uint32_t node_id) { + pw_init(nullptr, nullptr); + + std::cout << "Initializing PipeWire with fd: " << fd << ", node_id: " << node_id << std::endl; + + pw_loop_ = pw_thread_loop_new("screen-capture", nullptr); + if (!pw_loop_) { + std::cerr << "Failed to create Pipewire loop" << std::endl; + return false; + } + + pw_context_ = pw_context_new( + pw_thread_loop_get_loop(pw_loop_), + nullptr, 0 + ); + if (!pw_context_) { + std::cerr << "Failed to create Pipewire context" << std::endl; + pw_thread_loop_destroy(pw_loop_); + pw_loop_ = nullptr; + return false; + } + + // Connect to the PipeWire remote using the file descriptor + pw_core* core = pw_context_connect_fd( + pw_context_, + fd, + nullptr, 0 + ); + if (!core) { + std::cerr << "Failed to connect to PipeWire remote" << std::endl; + cleanupPipewire(); + return false; + } + + // Create stream properties + pw_properties* props = pw_properties_new( + PW_KEY_MEDIA_TYPE, "Video", + PW_KEY_MEDIA_CATEGORY, "Capture", + PW_KEY_MEDIA_ROLE, "Screen", + nullptr + ); + + // Create the stream + pw_stream_ = pw_stream_new( + core, + "screen-capture-stream", + props + ); + + if (!pw_stream_) { + std::cerr << "Failed to create PipeWire stream" << std::endl; + pw_core_disconnect(core); + cleanupPipewire(); + return false; + } + + // Set up stream events + static const pw_stream_events stream_events = { + PW_VERSION_STREAM_EVENTS, + nullptr, // destroy + nullptr, // state_changed + nullptr, // control_info + nullptr, // io_changed + onStreamParamChanged, // param_changed + nullptr, // add_buffer + nullptr, // remove_buffer + onStreamProcess, // process + nullptr, // drained + nullptr, // command + nullptr, // trigger_done + }; + + pw_stream_add_listener( + pw_stream_, + &stream_listener_, + &stream_events, + this // userdata pointer + ); + + // Connect to the node + uint8_t buffer[1024]; + spa_pod_builder pod_builder = SPA_POD_BUILDER_INIT(buffer, sizeof(buffer)); + + // Request any video format (we'll adapt to what we get) + const struct spa_pod* params[1]; + params[0] = reinterpret_cast(spa_pod_builder_add_object( + &pod_builder, + SPA_TYPE_OBJECT_Format, SPA_PARAM_EnumFormat, + SPA_FORMAT_mediaType, SPA_POD_Id(SPA_MEDIA_TYPE_video), + SPA_FORMAT_mediaSubtype, SPA_POD_Id(SPA_MEDIA_SUBTYPE_raw) + )); + + int result = pw_stream_connect( + pw_stream_, + PW_DIRECTION_INPUT, + node_id, + (pw_stream_flags)(PW_STREAM_FLAG_AUTOCONNECT | PW_STREAM_FLAG_MAP_BUFFERS), + params, 1 + ); + + if (result < 0) { + std::cerr << "Failed to connect stream to node " << node_id << ": " << strerror(-result) << std::endl; + pw_core_disconnect(core); + cleanupPipewire(); + return false; + } + + // Start the thread loop + if (pw_thread_loop_start(pw_loop_) < 0) { + std::cerr << "Failed to start PipeWire thread loop" << std::endl; + pw_core_disconnect(core); + cleanupPipewire(); + return false; + } + + std::cout << "PipeWire stream connected and running" << std::endl; + return true; +} + +void ScreenCapture::cleanupPipewire() { + if (pw_loop_) { + pw_thread_loop_stop(pw_loop_); + } + + if (pw_stream_) { + pw_stream_destroy(pw_stream_); + pw_stream_ = nullptr; + } + + if (pw_context_) { + pw_context_destroy(pw_context_); + pw_context_ = nullptr; + } + + if (pw_loop_) { + pw_thread_loop_destroy(pw_loop_); + pw_loop_ = nullptr; + } + + pw_deinit(); +} + +void ScreenCapture::onStreamProcess(void* userdata) { + auto* self = static_cast(userdata); + + pw_buffer* buffer = pw_stream_dequeue_buffer(self->pw_stream_); + if (!buffer) { + std::cerr << "No buffer available" << std::endl; + return; + } + + spa_buffer* spa_buf = buffer->buffer; + if (!spa_buf || !spa_buf->datas || !spa_buf->datas[0].data) { + pw_stream_queue_buffer(self->pw_stream_, buffer); + return; + } + + // Get frame data + spa_data& data = spa_buf->datas[0]; + uint8_t* frame_data = static_cast(data.data); + uint32_t size = data.chunk->size; + + // Invoke the callback if set + if (self->frameCallback_) { + // Convert to vector for callback + std::vector frame_vec(frame_data, frame_data + size); + self->frameCallback_(frame_vec, self->frame_width_, self->frame_height_); + } + + // Return buffer to stream + pw_stream_queue_buffer(self->pw_stream_, buffer); +} + +void ScreenCapture::onStreamParamChanged(void* userdata, uint32_t id, const struct spa_pod* param) { + auto* self = static_cast(userdata); + + if (!param || id != SPA_PARAM_Format) { + return; + } + + // Parse video format + spa_video_info_raw video_info; + if (spa_format_video_raw_parse(param, &video_info) < 0) { + std::cerr << "Failed to parse video format" << std::endl; + return; + } + + // Update our stored dimensions + self->frame_width_ = video_info.size.width; + self->frame_height_ = video_info.size.height; + + std::cout << "Stream format changed:" << std::endl; + std::cout << " Resolution: " << self->frame_width_ << "x" << self->frame_height_ << std::endl; + std::cout << " Format: " << video_info.format << std::endl; + std::cout << " Framerate: " << video_info.framerate.num << "/" << video_info.framerate.denom << std::endl; +} + +#else +// Stub implementations for non-Linux platforms +bool ScreenCapture::initPortalConnection() { return false; } +void ScreenCapture::cleanupPortalConnection() {} +std::string ScreenCapture::createPortalSession() { return ""; } +bool ScreenCapture::selectPortalSources(const std::string&) { return false; } +bool ScreenCapture::startPortalSession(const std::string&, uint32_t&) { return false; } +int ScreenCapture::openPipeWireRemote(const std::string&) { return -1; } +uint32_t ScreenCapture::getStreamsNodeId(const std::string&) { return 0; } +bool ScreenCapture::initPipewire(int, uint32_t) { return false; } +void ScreenCapture::cleanupPipewire() {} +void ScreenCapture::onStreamProcess(void*) {} +void ScreenCapture::onStreamParamChanged(void*, uint32_t, const struct spa_pod*) {} +#endif + } // namespace scar diff --git a/client/media/screen_capture.h b/client/media/screen_capture.h index 7450b99..dbbee05 100644 --- a/client/media/screen_capture.h +++ b/client/media/screen_capture.h @@ -5,6 +5,21 @@ #include #include +#ifdef __linux__ +#include +#include +#endif + +// Forward declarations for Pipewire/sdbus to avoid header pollution +struct pw_thread_loop; +struct pw_stream; +struct pw_context; + +namespace sdbus { + class IConnection; + class IProxy; +} + namespace scar { enum class ScreenCaptureBackend { @@ -44,18 +59,43 @@ private: bool startWindowsCapture(); bool startPortalCapture(); + // sdbus Portal methods + bool initPortalConnection(); + void cleanupPortalConnection(); + std::string createPortalSession(); + bool selectPortalSources(const std::string& session_handle); + bool startPortalSession(const std::string& session_handle, uint32_t& node_id); + int openPipeWireRemote(const std::string& session_handle); + uint32_t getStreamsNodeId(const std::string& session_handle); + + // Pipewire methods + bool initPipewire(int fd, uint32_t node_id); + void cleanupPipewire(); + static void onStreamProcess(void* userdata); + static void onStreamParamChanged(void* userdata, uint32_t id, const struct spa_pod* param); + bool capturing_; ScreenCaptureBackend backend_; FrameCallback frameCallback_; + // sdbus members + std::shared_ptr portal_connection_; + std::unique_ptr screencast_proxy_; + std::string session_path_; + + // Pipewire members + pw_thread_loop* pw_loop_; + pw_stream* pw_stream_; + pw_context* pw_context_; + spa_hook stream_listener_; + + int frame_width_; + int frame_height_; + // TODO: FFmpeg-specific members // AVFormatContext* formatCtx_; // AVCodecContext* codecCtx_; // AVFrame* frame_; - - // TODO: DBus/Portal-specific members for Wayland - // DBusConnection* dbusConn_; - // pw_stream* pipewireStream_; }; } // namespace scar