mirror of
https://gitlab.freedesktop.org/pipewire/pipewire
synced 2024-09-06 00:40:34 +00:00
150 lines
6.9 KiB
Plaintext
150 lines
6.9 KiB
Plaintext
/** \page page_dma_buf DMA-BUF sharing
|
|
|
|
PipeWire supports sharing Direct Memory Access buffers (DMA-BUFs) between
|
|
clients via the `SPA_DATA_DmaBuf` data type. However properly negotiating
|
|
DMA-BUF support on both the producer and the consumer side require following
|
|
a specific procedure. This page describes said procedure by using events and
|
|
methods from the filter or stream API.
|
|
Note: This article focuses mostly on DMA-BUF sharing from arbitrary devices,
|
|
like discrete GPUs. For using DMA-BUFs created by v4l2 please refer to the
|
|
corresponding paragraph.
|
|
|
|
# Capability negotiations
|
|
|
|
The capability negotiation for DMA-BUFs is complicated by the fact, that a
|
|
usable and preferred optimal modifier for a given format can only be
|
|
determined by the allocator, which has to be invoked with the intersection
|
|
of all supported modifiers of all clients. As a result the fixation of the
|
|
modifier has to be delegated from PipeWire to the node responsible for
|
|
allocating the buffers.
|
|
|
|
## pw_stream_connect
|
|
|
|
The stream parameters should contain two `SPA_PARAM_EnumFormat` objects for
|
|
each format: the first one is used for DMA-BUFs, the second one for shared
|
|
memory buffers as a fallback.
|
|
|
|
Query the list of all supported modifiers from your graphics API of choice.
|
|
Add a `SPA_FORMAT_VIDEO_modifier` property to the first stream parameter with
|
|
the flags `SPA_POD_PROP_FLAG_MANDATORY | SPA_POD_PROP_FLAG_DONT_FIXATE`. The
|
|
value of the property should be set to a `SPA_CHOICE_Enum` containing one
|
|
`long` choice per supported modifier, plus `DRM_FORMAT_MOD_INVALID` if the
|
|
graphics API supports modifier-less buffers.
|
|
|
|
Note: When a producer is only supporting modifier-less buffers it can omit
|
|
the `SPA_POD_PROP_FLAG_DONT_FIXATE` (see param_changed hook, For producers).
|
|
|
|
The second stream parameter should not contain any `SPA_FORMAT_VIDEO_modifier`
|
|
property.
|
|
|
|
To prioritise DMA-BUFs place those `SPA_PARAM_EnumFormat` containing modifiers
|
|
first, when emitting them to PipeWire.
|
|
|
|
## param_changed hook
|
|
|
|
When the `param_changed` hook is called for a `SPA_PARAM_Format` the client
|
|
has to parse the `spa_pod` directly. Use
|
|
`spa_pod_find_prop(param, NULL, SPA_FORMAT_VIDEO_modifier)` to check
|
|
whether modifiers were negotiated. If they were negotiated, set the
|
|
`SPA_PARAM_BUFFERS_dataType` property to `1 << SPA_DATA_DmaBuf`. If they were
|
|
not negotiated, fall back to shared memory by setting the
|
|
`SPA_PARAM_BUFFERS_dataType` property to `1 << SPA_DATA_MemFd`,
|
|
`1 << SPA_DATA_MemPtr`, or both.
|
|
|
|
While consumers only have to parse the resulting `SPA_PARAM_Format` for any
|
|
format related information, it's up to the producer to fixate onto a single
|
|
format modifier pair. The producer is also responsible to check if all clients
|
|
announce sufficient capabilities or fallback to shared memory buffers when
|
|
possible.
|
|
|
|
### For consumers
|
|
|
|
Use `spa_format_video_raw_parse` to get the format and modifier.
|
|
|
|
### For producers
|
|
|
|
Producers have to handle two cases when it comes to modifiers wrt. to the
|
|
previous announced capabilities: Using only the modifier-less API, only the
|
|
modifier aware one, or supporting both.
|
|
|
|
- modifier-less:
|
|
In this case only the modifier `DRM_FORMAT_MOD_INVALID` was announced with
|
|
the format.
|
|
It is sufficient to check if the `SPA_PARAM_Format` contains the modifier
|
|
property as described above. Is that the case use DMA-BUFs for screen-sharing,
|
|
else fall back to SHM, if possible.
|
|
- modifier-aware:
|
|
In this case a list with all supported modifiers will be returned in the format.
|
|
(using `DRM_FORMAT_MOD_INVALID` as the token for the modifier-less API).
|
|
On the `param_changed` event check if the modifier key is present and has the flag
|
|
`SPA_POD_PROP_FLAG_DONT_FIXATE`. attached to it. In this case extract all modifiers
|
|
from the list and do a test allocation with your allocator to choose the preferred
|
|
modifier. Fixate on that `EnumFormat` by announcing a `SPA_PARAM_EnumFormat` with
|
|
only one modifier in the `SPA_CHOICE_Enum` and without the
|
|
`SPA_POD_PROP_FLAG_DONT_FIXATE`, followed by the previous announced
|
|
`EnumFormat`s. This will retrigger (**TBD**) the `param_changed` event with an
|
|
`SPA_PARAM_Format` as described below.
|
|
If the `SPA_PARAM_Format` contains a modifier key, without the flag
|
|
`SPA_POD_PROP_FLAG_DONT_FIXATE`, it should only contain one value in the
|
|
`SPA_CHOICE_Enum`. In this case announce the `SPA_PARAM_Buffers` accordingly
|
|
to the selected format and modifier. It is important to query the plane count
|
|
of the used format modifier pair and set `SPA_PARAM_BUFFERS_blocks` accordingly.
|
|
|
|
Note: When test allocating a buffer, collect all possible modifiers, while omitting
|
|
`DRM_FORMAT_MOD_INVALID` from the `SPA_FORMAT_VIDEO_modifier` property and
|
|
pass them all to the graphics API. If the allocation fails and the list of
|
|
possible modifiers contains `DRM_FORMAT_MOD_INVALID`, fall back to allocating
|
|
without an explicit modifier if the graphics API allows it.
|
|
|
|
## add_buffer hook
|
|
|
|
This is relevant for producers.
|
|
|
|
Allocate a DMA-BUF only using the negotiated format and modifier.
|
|
|
|
## on_event hook
|
|
|
|
This is relevant for consumers.
|
|
|
|
Check the type of the dequeued buffer. If its `SPA_DATA_MemFd` or
|
|
`SPA_DATA_MemPtr` use the fallback SHM import mechanism.
|
|
If it's `SPA_DATA_DmaBuf`:
|
|
Get the DMA-BUF FDs (the plane count is encoded in the `n_datas` variable of the
|
|
`spa_buffer` struct) and import them with the graphics API.
|
|
|
|
Note: Some graphics APIs have separated functions for the modifier-less case
|
|
(`DRM_FORMAT_MOD_INVALID`) or are omitting the modifier, since it might be used
|
|
for error handling.
|
|
|
|
# DMA-BUF mapping warning
|
|
|
|
It's important to make sure all consumers of the PipeWire stream are prepared
|
|
to deal with DMA-BUFs. Most DMA-BUFs cannot be treated like shared memory in general
|
|
because of the following issues:
|
|
|
|
- DMA-BUFs can use hardware-specific tiling and compression as described by
|
|
modifiers. Thus, a `mmap(3)` on the DMA-BUF FD will not give a linear view of
|
|
the buffer contents.
|
|
- DMA-BUFs need to be properly synchronized with the asynchronous reads and
|
|
writes of the hardware. A `mmap(3)` call is not enough to guarantee proper
|
|
synchronization. (Maybe add link to linux syscall doc??)
|
|
- Blindly accessing the DMA-BUFs via `mmap(3)` can be extremely slow if the
|
|
buffer has been allocated on discrete hardware. Consumers are better off
|
|
using a proper graphics API (such as EGL, Vulkan or VA-API) to process the
|
|
DMA-BUFs.
|
|
|
|
# v4l2
|
|
|
|
Another use case for streaming via DMA-BUFs are exporting a camera feed from v4l2
|
|
as DMA-BUFs. Those are located in the main memory where it is possible to mmap them.
|
|
This should be done as follows: Neither producer nor consumer should announce a
|
|
modifier, but both should include `1 << SPA_DATA_DmaBuf` in the
|
|
`SPA_PARAM_BUFFERS_dataType` property. It's the the responsibility of the producer
|
|
while the `add_buffer` event to choose DMA-BUF as the used buffer type even though
|
|
no modifier is present, if it can guarantee, that the used buffer is mmapable.
|
|
|
|
Note: For now v4l2 uses planar buffers without modifiers. This is the reason for
|
|
this special case.
|
|
|
|
*/
|