Completely rework how settings plugin handle connections and how
NMSettings tracks the list of connections.
Previously, settings plugins would return objects of (a subtype of) type
NMSettingsConnection. The NMSettingsConnection was tightly coupled with
the settings plugin. That has a lot of downsides.
Change that. When changing this basic relation how settings connections
are tracked, everything falls appart. That's why this is a huge change.
Also, since I have to largely rewrite the settings plugins, I also
added support for multiple keyfile directories, handle in-memory
connections only by keyfile plugin and (partly) use copy-on-write NMConnection
instances. I don't want to spend effort rewriting large parts while
preserving the old way, that anyway should change. E.g. while rewriting ifcfg-rh,
I don't want to let it handle in-memory connections because that's not right
long-term.
--
If the settings plugins themself create subtypes of NMSettingsConnection
instances, then a lot of knowledge about tracking connections moves
to the plugins.
Just try to follow the code what happend during nm_settings_add_connection().
Note how the logic is spread out:
- nm_settings_add_connection() calls plugin's add_connection()
- add_connection() creates a NMSettingsConnection subtype
- the plugin has to know that it's called during add-connection and
not emit NM_SETTINGS_PLUGIN_CONNECTION_ADDED signal
- NMSettings calls claim_connection() which hocks up the new
NMSettingsConnection instance and configures the instance
(like calling nm_settings_connection_added()).
This summary does not sound like a lot, but try to follow that code. The logic
is all over the place.
Instead, settings plugins should have a very simple API for adding, modifying,
deleting, loading and reloading connections. All the plugin does is to return a
NMSettingsStorage handle. The storage instance is a handle to identify a profile
in storage (e.g. a particular file). The settings plugin is free to subtype
NMSettingsStorage, but it's not necessary.
There are no more events raised, and the settings plugin implements the small
API in a straightforward manner.
NMSettings now drives all of this. Even NMSettingsConnection has now
very little concern about how it's tracked and delegates only to NMSettings.
This should make settings plugins simpler. Currently settings plugins
are so cumbersome to implement, that we avoid having them. It should not be
like that and it should be easy, beneficial and lightweight to create a new
settings plugin.
Note also how the settings plugins no longer care about duplicate UUIDs.
Duplicated UUIDs are a fact of life and NMSettings must handle them. No
need to overly concern settings plugins with that.
--
NMSettingsConnection is exposed directly on D-Bus (being a subtype of
NMDBusObject) but it was also a GObject type provided by the settings
plugin. Hence, it was not possible to migrate a profile from one plugin to
another.
However that would be useful when one profile does not support a
connection type (like ifcfg-rh not supporting VPN). Currently such
migration is not implemented except for migrating them to/from keyfile's
run directory. The problem is that migrating profiles in general is
complicated but in some cases it is important to do.
For example checkpoint rollback should recreate the profile in the right
settings plugin, not just add it to persistent storage. This is not yet
properly implemented.
--
Previously, both keyfile and ifcfg-rh plugin implemented in-memory (unsaved)
profiles, while ifupdown plugin cannot handle them. That meant duplication of code
and a ifupdown profile could not be modified or made unsaved.
This is now unified and only keyfile plugin handles in-memory profiles (bgo #744711).
Also, NMSettings is aware of such profiles and treats them specially.
In particular, NMSettings drives the migration between persistent and non-persistent
storage.
Note that a settings plugins may create truly generated, in-memory profiles.
The settings plugin is free to generate and persist the profiles in any way it
wishes. But the concept of "unsaved" profiles is now something explicitly handled
by keyfile plugin. Also, these "unsaved" keyfile profiles are persisted to file system
too, to the /run directory. This is great for two reasons: first of all, all
profiles from keyfile storage in fact have a backing file -- even the
unsaved ones. It also means you can create "unsaved" profiles in /run
and load them with `nmcli connection load`, meaning there is a file
based API for creating unsaved profiles.
The other advantage is that these profiles now survive restarting
NetworkManager. It's paramount that restarting the daemon is as
non-disruptive as possible. Persisting unsaved files to /run improves
here significantly.
--
In the past, NMSettingsConnection also implemented NMConnection interface.
That was already changed a while ago and instead users call now
nm_settings_connection_get_connection() to delegate to a
NMSimpleConnection. What however still happened was that the NMConnection
instance gets never swapped but instead the instance was modified with
nm_connection_replace_settings_from_connection(), clear-secrets, etc.
Change that and treat the NMConnection instance immutable. Instead of modifying
it, reference/clone a new instance. This changes that previously when somebody
wanted to keep a reference to an NMConnection, then the profile would be cloned.
Now, it is supposed to be safe to reference the instance directly and everybody
must ensure not to modify the instance. nmtst_connection_assert_unchanging()
should help with that.
The point is that the settings plugins may keep references to the
NMConnection instance, and so does the NMSettingsConnection. We want
to avoid cloning the instances as long as they are the same.
Likewise, the device's applied connection can now also be referenced
instead of cloning it. This is not yet done, and possibly there are
further improvements possible.
--
Also implement multiple keyfile directores /usr/lib, /etc, /run (rh #1674545,
bgo #772414).
It was always the case that multiple files could provide the same UUID
(both in case of keyfile and ifcfg-rh). For keyfile plugin, if a profile in
read-only storage in /usr/lib gets modified, then it gets actually stored in
/etc (or /run, if the profile is unsaved).
--
While at it, make /etc/network/interfaces profiles for ifupdown plugin reloadable.
--
https://bugzilla.gnome.org/show_bug.cgi?id=772414https://bugzilla.gnome.org/show_bug.cgi?id=744711https://bugzilla.redhat.com/show_bug.cgi?id=1674545
The file got a wider scope to contain generic meta data about profiles.
Rename the internal API to reflect that (and be consistend with the
naming of the files).
We may want to store meta-data for a profile to disk. The immediate
need are "tombstones": markers that the particular UUID is shadowed
and the profile does not exist (despite being in read-only location).
Change the filename of these symlinks from
".loaded-${UUID}.nmconnection"
to
"${UUID}.nmmeta"
The leading dot is not desirable as tools tend to hide such files.
Use a different scheme for the filename that does not have the leading dot.
Note that nm_keyfile_utils_ignore_filename() would also ignore ".nmmeta"
as not a valid keyfile. This is just what we want, and influences the
choice of this file suffix.
Also, "nmmeta" is a better name, because this name alludes that there is
a wider use for the file: namely to have addtional per-profile metadata.
That is regardless that the upcoming first use will be only to store symlinks
to "/dev/null" to indicate the tombstones.
Note that per-profile metadata is not new. Currently we write the files
/var/lib/NetworkManager/{seen-bssids,timestamps}
that have a similar purpose. Maybe the content from these files could one
day be migrated to the ".nmmeta" file. The naming scheme would make it
suitable.
nm_strndup_a() uses strncpy() because we want the behavior of clearing out
the memory after the first NUL byte. But that can cause a compiler warning:
CC src/settings/plugins/keyfile/libNetworkManager_la-nms-keyfile-utils.lo
In file included from ../../shared/nm-default.h:279,
from ../../src/settings/plugins/keyfile/nms-keyfile-utils.c:20:
In function ‘_nm_strndup_a_step’,
inlined from ‘nms_keyfile_loaded_uuid_is_filename’ at ../../src/settings/plugins/keyfile/nms-keyfile-utils.c:65:9:
../../shared/nm-glib-aux/nm-macros-internal.h:1661:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
1661 | strncpy (s, str, len);
| ^~~~~~~~~~~~~~~~~~~~~
../../src/settings/plugins/keyfile/nms-keyfile-utils.c: In function ‘nms_keyfile_loaded_uuid_is_filename’:
../../src/settings/plugins/keyfile/nms-keyfile-utils.c:48:8: note: length computed here
48 | len = strlen (filename);
| ^~~~~~~~~~~~~~~~~
It's true that the len argument of _nm_strndup_a_step() depends on the
string length of the source string. But in this case it's safe, because
we checked that the destination buffer is exactly the right size too.
By that reasoning we should use memcpy() or strcpy(), but both are
unsuitable. That is because we want nm_strndup_a() to behave like
strndup(), which means we need to handle cases where the len argument
is larger than the string length of the source string. That is, we want
always to return a buffer of size len+1, but we want to copy only the
characters up to the first NUL byte, and clear out the rest. That's what
strncpy() does for us.
Silence the warning.
4 properties are not really relevant for an already activated connection
or it makes not sense to change them. These are connection.id, connection.uuid,
connection.autoconnect and connection.stable-id.
For convenience, we allow to reapply these. This way, one can take
a different setting (e.g. with a different connection.id or
connection.uuid) and reapply them, but such changes are silently
ignored.
However this was done wrongly. Instead of reverting the change to the new
applied connection, we would change the input connection.
This is bad, for example with
nmcli connection up uuid cb922f18-e99a-49c6-b200-1678b5070a82
nmcli connection modify cb922f18-e99a-49c6-b200-1678b5070a82 con-name "bogus"
nmcli device reapply eth0
the last re-apply would reset the settings-connection's connection ID to
what was before, while accepting the new name on the applied-connection
(while it should have been rejected).
Fixes: bf3b3d444c ('device: avoid changing immutable properties during reapply')
IP addresses, routes, TC and QDiscs are all tied to a certain interface.
So when NetworkManager manages an interface, it can be confident that
all related entires should be managed, deleted and modified by NetworkManager.
Routing policy rules are global. For that we have NMPRulesManager which
keeps track of whether NetworkManager owns a rule. This allows multiple
connection profiles to specify the same rule, and NMPRulesManager can
consolidate this information to know whether to add or remove the rule.
NMPRulesManager would also support to explicitly block a rule by
tracking it with negative priority. However that is still unused at
the moment. All that devices do is to add rules (track with positive
priority) and remove them (untrack) once the profile gets deactivated.
As rules are not exclusively owned by NetworkManager, NetworkManager
tries not to interfere with rules that it knows nothing about. That
means in particular, when NetworkManager starts it will "weakly track"
all rules that are present. "weakly track" is mostly interesting for two
cases:
- when NMPRulesManager had the same rule explicitly tracked (added) by a
device, then deactivating the device will leave the rule in place.
- when NMPRulesManager had the same rule explicitly blocked (tracked
with negative priority), then it would restore the rule when that
block gets removed (as said, currently nobody actually does this).
Note that when restarting NetworkManager, then the device may stay and
the rules kept. However after restart, NetworkManager no longer knows
that it previously added this route, so it would weakly track it and
never remove them again.
That is a problem. Avoid that, by whenever explicitly tracking a rule we
also make sure to no longer weakly track it. Most likely this rule was
indeed previously managed by NetworkManager. If this was really a rule
added by externally, then the user really should choose distinct
rule priorities to avoid such conflicts altogether.
WireGuard's wq-quick configures such rules to avoid routing loops.
While we currently don't have an automatic solution for this, at least
we should support it via explicit user configuration.
One problem is that suppress_prefixlength is relatively new and kernel
might not support this attribute. That can lead to odd results, because
the NetworkManager is valid but it cannot be configured on the current
kernel. But this is a general problem, and we would require a general
solution. The solution cannot be to only support rule attributes that
are supported by the oldest possible kernel. It's not clear how much of
a problem there really is, or which general solution is required (if
any).
The tables "main", "local", and "default" have well known names.
Accept them as aliases when parsing the string representation of
the rule.
Note that iproute2 also considers /etc/iproute2/rt_tables for table
names. In particular, that allows a user to re-map the well-known names
like "main" to a different table. We never honor that file, and "main"
always means table 254.
Note that this only affects how we parse the string representation for
rules. As the representation is neither unique nor enforced to be normalized,
being more graceful here is no problem.
The point is of course that the user possibly has existing iproute2
scripts that use such keyword. This makes it simpler to copy & paste
the rule.
When subscribing a signal to a singleton, we should ensure that the
source object stays alive. Take a reference.
This is also right in this case, because NMDBusManager (and its dependencies)
should never use NMDhcpListener. So, there is a clear direction of who references
who.
Our platform unit tests try to add an IP tunnel using iproute2.
That fails with
"add tunnel "ip6tnl0" failed: File exists"
This is a bug in iproute2-5.2.0, see [1].
Workaround the issue by downgrading the package.
[1] https://www.spinics.net/lists/netdev/msg584916.html
Rolling is the latest release (regardless of whether LTS), currently
that would be 19.04.
Devel is the next release, currently that would be 19.10.
Add manual build steps to trigger those builds so we can manually verify
that they pass.
Some tools that NM can interact with (eg. openconnect) have added
automated support to handle TPM2-wrapped PEM keys as drop-in
replacements for ordinary key files. Make sure that NM doesn't reject
these keys upfront. We cannot reliably assume NM to be able to unwrap
and validate the key. Therefore, accept any key as long as the PEM
header and trailer look ok.
Initscripts already honor the DEVTIMEOUT variable (rh #1171917).
Don't make this a property only supported by initscripts. Every
useful property should also be supported by keyfile and it should
be accessible via D-Bus.
Also, I will soon drop NMSIfcfgConnection, so handling this would
require extra code. It's easier when DEVTIMEOUT is a regular property of
the connection profile.
The property is not yet implemented. ifcfg-rh still uses the old
implementation, and keyfile is not yet adjusted. Since both keyfile
and ifcfg-rh will both be rewritten soon, this property will be
implemented then.
If there is only one argument, we can assume this is a plain string.
That is especially the case, because g_set_error() is G_GNUC_PRINTF()
and would warn if this would be a format string with missing parameters.
This is for convenience. Previously, one was compelled to explicitly
choose between nm_utils_error_set_literal() and nm_utils_error_set().
Now, it automatically chooses.
Note that there are a few things that won't work, like
nm_utils_error_set (error, code, "bogus %u escape");
But that's good. You get a compiler warning (as you used to)
and it's clear in this case you really need
nm_utils_error_set_literal().
This follows a pointer to a pointer and compares them. In a sense
it's like nm_pstr_*(), which follow a pointer to a string. However,
these functions use direct pointer comparison.
The purpose is when you hash a key that has as first field a pointer
value (and then compare them by pointer equality).
Seems that a quite common openswitch basic configuration consist of a
one bridge, one port and one interface, all with the same interface
name. When performing such configuration in NetworkManager you need to
specify the slave-type for the ovs-interface, otherwise the master
interface specified there may match the bridge interface, resulting in
an error.
So, let's specify the slave-type for the ovs-interface, so that the
example will work also when the same interface name is specified for
both the ovs-bridge and the ovs-port.
https://bugzilla.redhat.com/show_bug.cgi?id=1638038
nmtst: initialize nmtst_get_rand() with NMTST_SEED_RAND=0
/link/bogus: OK
/link/loopback: OK
/link/internal: OK
/link/external: OK
/link/software/bridge: OK
/link/software/bond: OK
/link/software/team: NMPlatformSignalAssert: ../src/platform/tests/test-link.c:331, test_slave(): failure to accept signal [0,2] times: 'link-changed-changed' ifindex 15 (3 times received)
--- stderr ---
/builds/NetworkManager/NetworkManager/tools/run-nm-test.sh: line 264: 106682 Trace/breakpoint trap --quiet --error-exitcode= --leak-check=full --gen-suppressions=all --num-callers=100 --log-file=
The test failed. Also check the valgrind log at '/builds/NetworkManager/NetworkManager/build/src/platform/tests/test-link-linux.valgrind-log'
In file included from ./shared/systemd/sd-adapt-shared/nm-sd-adapt-shared.h:21,
from shared/systemd/src/shared/dns-domain.c:3:
./shared/nm-default.h:106:10: fatal error: config-extra.h: No such file or directory
#include "config-extra.h"
^~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [Makefile:12933: shared/systemd/src/shared/libnm_systemd_shared_la-dns-domain.lo] Error 1
Fixes: 7d3098ff90 ('systemd: add dns-domain utils to systemd static library')
If the master is activated by user, propagate the for-user-request to
slaves activations when autoconnecting slaves, so that they can manage
slaves device as needed.
Reproducer:
ip l add eth1 type veth peer name eth2
ip l set eth1 up
ip l set eth2 up
sleep 2
echo " * Initial state"
echo " - eth1: $(nmcli -g general.state device show eth1)"
nmcli con add type ethernet ifname eth1 con-name slave-test+ master br-test slave-type bridge
nmcli con add type bridge ifname br-test con-name br-test+ connection.autoconnect-slaves yes ip4 172.25.1.1/24
nmcli con up br-test+
echo " * After user activation"
echo " - br-test: $(nmcli -g general.state device show br-test)"
echo " - eth1: $(nmcli -g general.state device show eth1)"
should give:
* Initial state
- eth1: 10 (unmanaged)
* After user activation
- br-test: 100 (connected)
- eth1: 100 (connected)
The previous code returned that the device was available when it had
only unmanaged-flags that can be overridden by user, without actually
considering the @flags argument.
Fixes: 920346a5b9 ('device: add and use overrule-unmanaged flag for nm_device_check_connection_available()')
Some linux distros (e.g., RHEL, CentOS and Fedora) ship a dispatcher
script with their dhclient package in order to run user dhclient hook
scripts also when connections are run by NetworkManager.
The scripts convert the var names in the environment to the ones
expected in the dhclient hook scripts.
This feature is currently use by cloud-init to retrieve the MS Azure
server endpoint address, which is sent in the private dhcp option 245.
We just redefined the default dhclient var names for the private options
from "unknown_xyz" to "private_xyz". In order to let current cloud-init
version to be able to retrieve the Azure server endpoint address, add
the legacy var name "unknown_245" to the dispatcher script environment.
when dhclient is used as the dhcp client in NetworkManager we expose on
D-Bus all the variables that are passed to our script file. In
particular, we use the variable names there as labels (stripping the
heading "new") taking whatever dhclient passes us.
There are few exception to this. Dhclient allows to redefine option
variable names and we use this functionality for a few dhcp options:
dhcp option code 121 --> "rfc3442_classless_static_routes"
dhcp option code 249 --> "ms_classless_static_routes"
dhcp option code 252 --> "wpad"
Note that for private dhcp options (224-254) default dhclient labels are
in the form "unknown_$OPTNUM".
When using the internal dhcp client we skip exporting on D-Bus many of
the dhcp options received from the dhcp server. We instead export almost
all of them when using the dhclient dhcp client, using the variable
names passed by dhclient itself.
Map more DHCP options to dhclient variable names in order to allow the
internal client to retrieve them easily, namely: the server identifier,
the broadcast address, the renewal time, the rebinding time and the timezone.
Note that not all the DHCP options can be exported at this time because
systemd-networkd code drops many it won't process, so we have no way to
retrieve them without changing core systemd-networkd code.
