mirror of
https://gitlab.freedesktop.org/pipewire/pipewire
synced 2024-07-20 17:55:41 +00:00
doc: add midi doc
Reorganize the docs a little. First a short intro, then list the use cases, then the responsabilities of the various components, then the implementation in various places.
This commit is contained in:
Wim Taymans
parent
76cb5c1169
commit
0346c616e2
|
|
@ -2,6 +2,57 @@
|
|||
|
||||
This document explains how access control is designed implemented.
|
||||
|
||||
PipeWire implements per client permissions on the objects in the graph.
|
||||
|
||||
Permissions include R (read), W (write), X (execute) and M (metadata).
|
||||
|
||||
An object with R permissions is visible to the client. The client will
|
||||
receive registry events for the object and can interact with it.
|
||||
|
||||
Methods can be called on an object with X permissions. Some of those
|
||||
methods will only query state, others will modify the object.
|
||||
|
||||
An object with W permission can be modified. This is usually done with
|
||||
a method that modifies the state of the object. The W permission usually
|
||||
implies the X permission.
|
||||
|
||||
An object with M permission can be used as the subject in metadata.
|
||||
|
||||
# Use cases
|
||||
|
||||
## new clients need their permissions configured
|
||||
|
||||
A new client is not allowed to communicate with the PipeWire daemon until
|
||||
it has been configured with permissions.
|
||||
|
||||
## Flatpaks can't modify other stream/device volumes
|
||||
|
||||
Flatpaks should not be able to modify the state of certain objects.
|
||||
|
||||
Permissions of the relevant PipeWire objects should not have the W
|
||||
permission to avoid this.
|
||||
|
||||
## Flatpaks can't move other streams to different devices
|
||||
|
||||
Streams are moved to another device by setting the "target.node" metadata
|
||||
on the node id. By not setting the M bit on the other objects, this can
|
||||
be avoided.
|
||||
|
||||
## Some application should be restricted in what they can see
|
||||
|
||||
Application should only be able to see the objects that they are allowed
|
||||
to see. For example, a web browser that was given access to a camera should
|
||||
not be able to see audio devices.
|
||||
|
||||
## Manager applications
|
||||
|
||||
Some applications might be shipped in a flatpak but really require full
|
||||
access to the PipeWire graph. This includes moving streams
|
||||
(setting metadata) and modifying properties (volume).
|
||||
|
||||
|
||||
# Design
|
||||
|
||||
## The PipeWire daemon
|
||||
|
||||
When a new client connects to the PipeWire daemon and right after it
|
||||
|
|
@ -84,32 +135,7 @@ Permissions of objects for a client can be changed at any time by the
|
|||
session manager. Removing the client core R permission will suspend the
|
||||
client completely.
|
||||
|
||||
|
||||
## Use cases
|
||||
|
||||
### new clients need their permissions configured
|
||||
|
||||
A new client that has been suspended by the PipeWire access module need
|
||||
to be resumed at some point with the right permissions.
|
||||
|
||||
### Flatpaks can't modify other stream/device volumes
|
||||
|
||||
Permissions of the PipeWire objects should not have the W permission to
|
||||
avoid this.
|
||||
|
||||
### Flatpaks can't move other streams to different devices
|
||||
|
||||
Streams are moved to another device by setting the "target.node" metadata
|
||||
on the node id. By not setting the M bit on the other objects, this can
|
||||
be avoided.
|
||||
|
||||
### Manager applications
|
||||
|
||||
Some applications might be shipped in a flatpak but really require full
|
||||
access to the PipeWire graph. This includes moving streams
|
||||
(setting metadata) and modifying properties (volume).
|
||||
|
||||
These manager applications will set the PW_KEY_MEDIA_CATEGORY property
|
||||
Manager applications will set the PW_KEY_MEDIA_CATEGORY property
|
||||
in the client object to "Manager".
|
||||
|
||||
The session manager needs to do additional checks to determine if the
|
||||
|
|
@ -119,7 +145,9 @@ permission store checks or ask the user if the application is allowed
|
|||
full access.
|
||||
|
||||
|
||||
### pipewire-media-session
|
||||
# Implementation
|
||||
|
||||
## pipewire-media-session
|
||||
|
||||
The example media session has an access-flatpak module that handles the
|
||||
clients that have a PW_KEY_ACCESS as "flatpak". Other clients are
|
||||
|
|
|
|||
102
doc/pipewire-midi.dox
Normal file
102
doc/pipewire-midi.dox
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
/** \page page_midi PipeWire MIDI
|
||||
|
||||
This document explains how MIDI is implemented.
|
||||
|
||||
# Use cases
|
||||
|
||||
## MIDI devices are made available as processing nodes/ports
|
||||
|
||||
Applications need to be able to see a port for each stream of a
|
||||
MIDI device.
|
||||
|
||||
## MIDI devices can be plugged and unplugged
|
||||
|
||||
When devices are plugged and unplugged the associated nodes/ports
|
||||
need to be created and removed.
|
||||
|
||||
## Applications can connect to MIDI devices
|
||||
|
||||
Applications can create ports that can connect to the MIDI ports
|
||||
so that data can be provided to or consumed from them.
|
||||
|
||||
## Some MIDI devices are sinks or sources for midi data
|
||||
|
||||
It should be possible to create a MIDI sink or source that routes the
|
||||
midi events to specific midi ports.
|
||||
|
||||
One example of such a sink would be in front of a software midi
|
||||
renderer.
|
||||
|
||||
An example of a MIDI source would be after a virtual keyboard or
|
||||
as a mix from many midi input devices.
|
||||
|
||||
## Applications should autoconnect to MIDI sinks or sources
|
||||
|
||||
An application should be able to be connected to a MIDI sink when
|
||||
it wants to play midi data.
|
||||
|
||||
An application should be able to connect to a MIDI source when it
|
||||
wants to capture midi data.
|
||||
|
||||
# Design
|
||||
|
||||
## SPA
|
||||
|
||||
MIDI devices/streams are implemented with an SPA Node with generic
|
||||
control input and output Ports. These ports have a media type of
|
||||
"application/control" and the data transported over these ports
|
||||
are of type spa_pod_sequence with the spa_pod_control type set to
|
||||
SPA_CONTROL_Midi.
|
||||
|
||||
This means that every midi event is timestamped with the sample
|
||||
offset against the current graph clock cycle to get sample accurate
|
||||
midi events that can be aligned with the corresponding sample data.
|
||||
|
||||
Since the MIDI events are embedded in the generic control stream,
|
||||
they can be interleaved with other control message types, such as
|
||||
property updates or OSC messages.
|
||||
|
||||
## The PipeWire daemon
|
||||
|
||||
Nothing special is implemented for MIDI. Negotiation of formats
|
||||
happens between "application/control" media types and buffers are
|
||||
negotiated in the same way as any generic format.
|
||||
|
||||
## The session manager
|
||||
|
||||
The session manager needs to create the MIDI nodes/ports for the available
|
||||
devices.
|
||||
|
||||
This can either be done as a single node with ports per device/stream
|
||||
or as separate nodes created by a MIDI device monitor.
|
||||
|
||||
The session manager needs to be aware of the various MIDI sinks and sources
|
||||
in order to route MIDI streams to them from applications that want this.
|
||||
|
||||
|
||||
# Implementation
|
||||
|
||||
## pipewire-media-session
|
||||
|
||||
PipeWire media session uses the SPA_NAME_API_ALSA_SEQ_BRIDGE plugin
|
||||
for the midi features. This creates a single SPA Node with ports per
|
||||
MIDI client/stream.
|
||||
|
||||
The media session will check the permissions on /dev/snd/seq before
|
||||
attempting to create this node. It will also use inotify to wait
|
||||
until the sequencer device node is accessible.
|
||||
|
||||
## JACK
|
||||
|
||||
JACK assumes all "application/control" ports are midi ports.
|
||||
|
||||
The control messages are converted to the JACK event format by
|
||||
filtering out the SPA_CONTROL_Midi types. On output ports, the JACK
|
||||
event stream is converted to control messages in a similar way.
|
||||
|
||||
There is a 1 to 1 mapping between the JACK events and control
|
||||
messages so there is no information loss or need for complicated
|
||||
conversions.
|
||||
|
||||
|
||||
*/
|
||||
|
|
@ -2,24 +2,55 @@
|
|||
|
||||
This document explains how clients from the portal are handled.
|
||||
|
||||
## The portal
|
||||
|
||||
The portal is a DBus service that exposes a couple of interfaces to
|
||||
request access to the PipeWire daemon to perform a certain set of
|
||||
functions.
|
||||
|
||||
The portal makes it possible to use PipeWire without having to expose
|
||||
the local PipeWire socket in the sandboxed application. PipeWire can
|
||||
detect and enforce extra permission checks on the Portal session.
|
||||
the local PipeWire socket in the sandboxed application. It is the
|
||||
portal can connects to PipeWire on behalf of the client. The client
|
||||
becomes a portal managed client. PipeWire can detect and enforce
|
||||
extra permission checks on the portal managed clients.
|
||||
|
||||
Once such portal is the Camera portal that provides a PipeWire session
|
||||
to stream video from a camera.
|
||||
|
||||
In the portal case, it is the portal itself that will make a connection
|
||||
to the PipeWire daemon to configure the session. It then hands the
|
||||
file descriptor of the PipeWire connection to the client. The client
|
||||
can then use the file descriptor to interface with the PipeWire
|
||||
session.
|
||||
# Use cases
|
||||
|
||||
## new portal managed clients need device permissions configured
|
||||
|
||||
When a new client is detected, the available objects need to be
|
||||
scanned and permissions configured for each of them.
|
||||
|
||||
Only the devices belonging to the media_roles given by the
|
||||
portal are considered.
|
||||
|
||||
## new devices need to be made visible to portal managed clients
|
||||
|
||||
Newly created objects are made visible to a client when the client
|
||||
is allowed to interact with it.
|
||||
|
||||
Only the devices belonging to the media_roles given by the
|
||||
portal are considered.
|
||||
|
||||
## permissions for a device need to be revoked
|
||||
|
||||
The session manager listens to changes in the permissions of devices
|
||||
and will remove the client permissions accordingly.
|
||||
|
||||
Usually this is implemented by listening to the permission store
|
||||
DBus object. The desktop environment might provide a configuration panel
|
||||
where these permissions can be managed.
|
||||
|
||||
|
||||
# Design
|
||||
|
||||
## The portal
|
||||
|
||||
The portal itself will make a connection to the PipeWire daemon to
|
||||
configure the session. It then hands the file descriptor of the PipeWire
|
||||
connection to the client. The client can then use the file descriptor
|
||||
to interface with the PipeWire session.
|
||||
|
||||
When the portal connects, it will set the following properties on the
|
||||
client object:
|
||||
|
|
@ -70,33 +101,7 @@ allowed.
|
|||
The permission store can be used for this. Usually the portal also
|
||||
implements "org.freedesktop.impl.portal.PermissionStore" for this.
|
||||
|
||||
|
||||
## Use cases
|
||||
|
||||
### new portal managed clients need device permissions configured
|
||||
|
||||
When a new client is detected, the available objects are scanned and
|
||||
permissions are configured for each of them.
|
||||
|
||||
Only the devices belonging to the media_roles given by the
|
||||
portal are considered.
|
||||
|
||||
### new devices need to be made visible to portal managed clients
|
||||
|
||||
Newly created objects are made visible to a client when the client
|
||||
is allowed to interact with it.
|
||||
|
||||
Only the devices belonging to the media_roles given by the
|
||||
portal are considered.
|
||||
|
||||
### permissions for a device need to be revoked
|
||||
|
||||
The session manager listens to changes in the permissions of devices
|
||||
and will remove the client permissions accordingly.
|
||||
|
||||
Usually this is implemented by listening to the permission store
|
||||
DBus object. The desktop environment might provide a configuration panel
|
||||
where these permissions can be managed.
|
||||
# Implementation
|
||||
|
||||
## pipewire-media-session
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue