From the files under "shared/nm-utils" we build an internal library
that provides glib-based helper utilities.
Move the files of that basic library to a new subdirectory
"shared/nm-glib-aux" and rename the helper library "libnm-core-base.la"
to "libnm-glib-aux.la".
Reasons:
- the name "utils" is overused in our code-base. Everything's an
"utils". Give this thing a more distinct name.
- there were additional files under "shared/nm-utils", which are not
part of this internal library "libnm-utils-base.la". All the files
that are part of this library should be together in the same
directory, but files that are not, should not be there.
- the new name should better convey what this library is and what is isn't:
it's a set of utilities and helper functions that extend glib with
funcitonality that we commonly need.
There are still some files left under "shared/nm-utils". They have less
a unifying propose to be in their own directory, so I leave them there
for now. But at least they are separate from "shared/nm-glib-aux",
which has a very clear purpose.
We built (among others) two libraries from the sources in "shared/nm-utils":
"libnm-utils-base.la" and "libnm-utils-udev.la".
It's confusing. Instead use directories so there is a direct
correspondence between these internal libraries and the source files.
"shared/nm-utils" contains general purpose utility functions that only
depend on glib (and extend glib with some helper functions).
We will also add code that does not use glib, hence it would be good
if the part of "shared/nm-utils" that does not depend on glib, could be
used by these future projects.
Also, we use the term "utils" everywhere. While that covers the purpose
and content well, having everything called "nm-something-utils" is not
great. Instead, call this "nm-std-aux", inspired by "c-util/c-stdaux".
This removes libnm-glib, libnm-glib-vpn, and libnm-util for good.
The it has been replaced with libnm since NetworkManager 1.0, disabled
by default since 1.12 and no up-to-date distributions ship it for years
now.
Removing the libraries allows us to:
* Remove the horrible hacks that were in place to deal with accidental use
of both the new and old library in a single process.
* Relief the translators of maintenance burden of similar yet different
strings.
* Get rid of known bad code without chances of ever getting fixed
(libnm-glib/nm-object.c and libnm-glib/nm-object-cache.c)
* Generally lower the footprint of the releases and our workspace
If there are some really really legacy users; they can just build
libnm-glib and friends from the NetworkManager-1.16 distribution. The
D-Bus API is stable and old libnm-glib will keep working forever.
https://github.com/NetworkManager/NetworkManager/pull/308
- use cleanup macros everywhere.
- In particular use nm_auto_clear_variant_builder to free the
GVariantBuilder in the error cases. Note that the error cases
anyway are asserted against, so during a normal test run there
was no leak. But we should not write software like that.
- use nm_utils_strsplit_set_with_empty() instead of g_strsplit_set().
We should use our variant also in unit-tests, because that way the
function gets more test coverage. And it likely performs better
anyway.
When nmcli needs secrets for a connection it asks them for every known
setting. nmtui is a bit smarter and asks them only for settings that
actually exist in the connection. Make a step further and let clients
ask secrets only for setting that exist *and* have any secret
property. This decreases the number of D-Bus calls when editing or
showing a connection with secrets.
https://bugzilla.redhat.com/show_bug.cgi?id=1506536https://github.com/NetworkManager/NetworkManager/pull/327
initscripts support rule-* and rule6-* files for that.
Up until now, we ignored these files for the most part, except if
a user configured such files, the profile could not contain any static
routes (or specify a route-table setting). This also worked together
with the dispatcher script "examples/dispatcher/10-ifcfg-rh-routes.sh".
We cannot now start taking over that file format for rules. It might
break existing setups, because we can never fully understand all rules as
they are understood by iproute2. Also, if a user has a rule/rule6 file and
uses NetworkManager successfully today, then clearly there is a script
in place to make that work. We must not break that when adding rules
support.
Hence, store routing rules as numbered "ROUTING_RULE_#" and
"ROUTING_RULE6_#" keys.
Note that we use different keys for IPv4 and IPv6. The main reason is
that the string format is mostly compatible with iproute2. That means,
you can take the value and pass it to `ip rule add`.
However, `ip rule add` only accepts IPv4 rules. For IPv6 rules, the user
needs to call `ip -6 rule add`. If we would use the same key for IPv4
and IPv6, then it would be hard to write a script to do this.
Also, nm_ip_routing_rule_from_string() does take the address family as
hint in this case. This makes
ROUTING_RULE_1="pref 1"
ROUTING_RULE6_1="pref 1"
automatically determine that address families. Otherwise, such
abbreviated forms would be not valid.
We have code in "shared/nm-utils" which are general purpose
helpers, independent of "libnm", "libnm-core", "clients" and "src".
We have shared code like "shared/nm-ethtool-utils.h" and
"shared/nm-meta-setting.h", which is statically linked, shared
code that contains libnm related helpers. But these helpers already
have a specific use (e.g. they are related to ethtool or NMSetting
metadata).
Add a general purpose helper that:
- depends (and extends) libnm-core
- contains unrelated helpers
- can be shared (meaning it will be statically linked).
- this code can be used by any library user of "libnm.so"
(nmcli, nm-applet) and by "libnm-core" itself. Thus, "src/"
and "libnm/" may also use this code indirectly, via "libnm-core/".
This removes libnm-glib, libnm-glib-vpn, and libnm-util for good.
The it has been replaced with libnm since NetworkManager 1.0, disabled
by default since 1.12 and no up-to-date distributions ship it for years
now.
Removing the libraries allows us to:
* Remove the horrible hacks that were in place to deal with accidental use
of both the new and old library in a single process.
* Relief the translators of maintenance burden of similar yet different
strings.
* Get rid of known bad code without chances of ever getting fixed
(libnm-glib/nm-object.c and libnm-glib/nm-object-cache.c)
* Generally lower the footprint of the releases and our workspace
If there are some really really legacy users; they can just build
libnm-glib and friends from the NetworkManager-1.16 distribution. The
D-Bus API is stable and old libnm-glib will keep working forever.
https://github.com/NetworkManager/NetworkManager/pull/308
Routing rules are unlike addresses or routes not tied to an interface.
NetworkManager thinks in terms of connection profiles. That works well
for addresses and routes, as one profile configures addresses and routes
for one device. For example, when activating a profile on a device, the
configuration does not interfere with the addresses/routes of other
devices. That is not the case for routing rules, which are global, netns-wide
entities.
When one connection profile specifies rules, then this per-device configuration
must be merged with the global configuration. And when a device disconnects later,
the rules must be removed.
Add a new NMPRulesManager API to track/untrack routing rules. Devices can
register/add there the routing rules they require. And the sync method will
apply the configuration. This is be implemented on top of NMPlatform's
caching API.
CC clients/nm_online-nm-online.o
In file included from ./shared/nm-default.h:311:0,
from clients/nm-online.c:34:
./libnm/NetworkManager.h:60:10: fatal error: nm-enum-types.h: No such file or directory
#include "nm-enum-types.h"
^~~~~~~~~~~~~~~~~
For now only add the core settings, no peers' data.
To support peers and the allowed-ips of the peers is more complicated
and will be done later. It's more complicated because these are nested
lists (allowed-ips) inside a list (peers). That is quite unusual and to
conveniently support that in D-Bus API, in keyfile format, in libnm,
and nmcli, is a effort.
Also, it's further complicated by the fact that each peer has a secret (the
preshared-key). Thus we probably need secret flags for each peer, which
is a novelty as well (until now we require a fixed set of secrets per
profile that is well known).
We named the types inconsistently:
- "p2p-wireless" ("libnm-core/nm-setting-p2p-wireless.h")
- "p2p" ("libnm/nm-p2p-peer.h")
- "p2p-wifi" ("src/devices/wifi/nm-device-p2p-wifi.h")
It seems to me, "libnm/nm-p2p-peer.h" should be qualified with a "Wi-Fi"
specific name. It's not just peer-to-peer, it's Wi-Fi P2P.
Yes, there is an inconsistency now, because there is already
"libnm/nm-access-point.h".
It seems to me (from looking at the internet), that the name "Wi-Fi P2P"
is more common than "P2P Wi-Fi" -- although both are used. There is also
the name "Wi-Fi Direct". But it's not clear which name should be
preferred here, so stick to "Wi-Fi P2P".
In this first commit only rename the files. The following commit will
rename the content.
It's not yet used, but it will be. We will need nm_sd_utils_unbase64mem()
to strictly validate WireGuard settings, which contain keys in base64 encoding.
Note that we also need a stub implementation for logging. This will do
nothing for all logging from "libnm-systemd-shared.a". This makes
sense because "libnm.so" as a library should not log directly. Also,
"libnm.so" will only use a small portion of "libnm-systemd-shared.a" which
doesn't log anything. Thus this code is unused and dropped by the linker
with "--gc-sections".
For better or worse, we already pull in large parts of systemd sources.
I need a base64 decode implementation (because glib's g_base64_decode()
cannot reject invalid encodings). Instead of coming up with my own or
copy-paste if from somewhere, reuse systemd's unbase64mem().
But for that, make systemd's basic bits an independent static library
first because I will need it in libnm-core.
This doesn't really change anything except making "libnm-systemd-core.la"
an indpendent static library that could be used from "libnm-core". We
shall still be mindful about which internal code of systemd we use, and only
access functionality that is exposed via "systemd/nm-sd-utils-shared.h".
We already import systemd code which is C11. To get this even
to build, we need workaround like patching import of <uchar.h>.
Also, the libraries from c-util and nettools are C11. We cannot even
compile them in C99 mode (and didn't do that either).
It's time to bump the version. We need C11 from now on (or better: gcc's
dialect of it).
Also, note that since nettools/nacd is not optional, we could not even
build NetworkManager without a C11 compiler. So, just use it everywhere.
In core ("src/"), we use "nm-logging.h" for all logging. This dispatches
for logging to syslog, glog or systemd-journald.
If we want to log from a shared component under "shared/", we need to
use a common logging function. Add "nm-utils/nm-logging-fwd.h" for
forward declaring the used logging mechaism.
The shared library will still need to link with "src/nm-logging.c"
or an alternative implementation, depending on whether it is used
inside core or not.
This will be our extension on top of <errno.h>.
We want to use (integer) error numbers, that can both
contain native errors from <errno.h> and our own defines,
both merge in one domain. That is, we will reserve a small
range of integers for our own defines (that hopefully won't
clash with errors from <errno.h>).
We can use this at places where GError is too cumbersome to use.
The advantage is, that our error numbers extend <errno.h> and can
be mixed.
This is what "src/platform/nm-netlink.h" already does with nl_errno(). Next,
the netlink errors from there will be merged into "nm-errno.h".
Also, platform has NMPlatformError, which are a distinct set of error
numbers. But these work differently in the sense that negative values
represent codes from <errno.h> and positive numbers are our own platform
specific defines. NMPlatformError will also be merged into "nm-errno.h".
"nm-errno.h" will unify the error handling of platform and netlink,
making it more similar to what we are used to from systemd, and give
room to extend it for our own purpose.
For P2P connections it makes sense to bind the connection to the status
of the operation that is being done. One example is that a wifi display
(miracast) P2P connection should be shut down when streaming fails for
some reason.
This new helper class allows binding a connection to the presence of a
DBus path meaning that it will be torn down if the process disappears.
Previously we would compile source files from shared/nm-utils
multiple times. That not only slows down compilation, but it makes it
confusing which project require exactly what.
Most of the files in shared/nm-utils are a mixed bag of utility
functions. Just build one libnm-utils-base library. Since the linker
will throw away unused parts, there is no problem that not every user
of libnm-utils-base needs everything.
Also add libnm-utils-udev, which cannot be part of libnm-utils-base as
it has an additional dependency on libudev.
Quote from `man NetworkManager.conf`:
When the default wired connection is deleted or saved to a new
persistent connection by a plugin, the device is added to a list in the
file /run/NetworkManager/no-auto-default.state to prevent creating
the default connection for that device again.
"/run" is obviously wrong. Fix it.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/33
We have a fork of a lot of useful systemd helper code.
However, until now we shyed away from using it aside from
the bits that we really need.
That means, although we have some really nice implementations
in our source-tree, we didn't use them. Either we were missing
them, or we had to re-implement them.
Add "nm-sd-utils.h" header to very carefully make internal
systemd API accessible to the rest of core.
This is not intended as a vehicle to access all of internal
API. Instead, this must be used with care, and only a hand picked
selection of functions must be exposed. Use with caution, but where it
makes sense.
"shared/nm-utils" is a loose collection of utility functions.
There is a certain aim that they can be used independently.
However, they also rely on each other.
Add a test that we can build a minimal shared library with
these tools, independent of libnm-core.
This is independent functionality that only depends on linux API
and glib.
Note how "nm-logging" uses this for getting the timestamps. This
makes "nm-logging.c" itself dependen on "src/nm-core-utils.c",
for little reason.
If the NetworkManager daemon has been stopped manually we don't want it
to be autostarted by a client request.
[lkundrak@v3.sk: The auto-activation is probably more surprising than useful.
Services that need NetworkManager API should depend on NetworkManager service
directly.
I have no idea what purpose does the D-Bus service file serve nowadays,
but it looks rather hacky (really, activating /bin/false) and the comment
in it suggests that the autoactivating behavior was not intended anyway.
Debian has been shipping this for quite some time and no complains have been
heard.]
https://github.com/NetworkManager/NetworkManager/pull/230
NMAcdManager is a rather simple instance.
It does not need (thread-safe) ref-counting, in fact, having
it ref-counted makes it slighly ugly that we connect a signal,
but never bother to disconnect it (while the ref-counted instance
could outlife the signal subscriber).
We also don't need GObject signals. They have more overhead
and are less type-safe than a regular function pointers. Signals
would make sense, if there could be multiple independent listeners,
but that just doesn't make sense.
Implementing it as a plain struct is less lines of code, and less
runtime over head.
Also drop the possiblitiy to reset the NMAcdManager instance.
It wasn't needed and I think it was buggy because it wouldn't
reset the n-acd instance.
https://github.com/NetworkManager/NetworkManager/pull/213
Adapt the meson post-installation script to handle the $DESTDIR
variable supplied by user to specify the installation target
directory. While at it, convert the script to shell because it seems
simpler to me.
nm-initrd-generator scans the command line for options relevant to network
configuration and creates configuration files for an early instance of
NetworkManager run from the initial ramdisk during early boot.
This is loosely based on nms-ibft-reader, but with some significant
changes. Notably, it parses /sys/firmware/ibft directly instead of
iscsiadm output.
iscsiadm is not available on early boot (perhaps it's too large) and
turns out that parsing sysfs directly is easier and more
straightforwared anyways. A win-win situation.
It is not useful alone, it's in a separate commit just for the sake of
easier review.
Add a configure option to disable eBPF support in n-acd.
Note that, even if eBPF is not supported, n-acd requires a kernel >
3.19, which means that the setsockopt(..., SO_ATTACH_BPF) option must
be defined. To allow building on older kernels without modifying the
n-acd code, we inject the SO_ATTACH_BPF value as a preprocessor define
in the compiler the command line.
Some path variable like $(bindir), $(datadir), etc. are special for
autotools and must be handled separately through config-extra.h.
But dhcp path variables are just normal variables defined through
the configure script and should go into config.h.
(cherry picked from commit 087c367d62)
If the library is available, let's at least compile both
crypto backends.
That is helpful when developing on crypto backends, so that
one does not have to configure the build twice.
With autotools, the build is only run during `make check`.
Not for meson, but that is generally the case with our meson
setup, that it also builds tests during the regular build step.
There are two aspects: the public crypto API that is provided by
"nm-crypto.h" header, and the internal header which crypto backends
need to implement. Split them.
We already had nm_free_secret() to clear the secret out
of a NUL terminated string. That works well for secrets
which are strings, it can be used with a cleanup attribute
(nm_auto_free_secret) and as a cleanup function for a
GBytes.
However, it does not work for secrets which are binary.
For those, we must also track the length of the allocated
data and clear it.
Add two new structs NMSecretPtr and NMSecretBuf to help
with that.
Add a new 'match' setting containing properties to match a connection
to devices. At the moment only the interface-name property is present
and, contrary to connection.interface-name, it allows the use of
wildcards.
Note that in NetworkManager API (D-Bus, libnm, and nmcli),
the features are called "feature-xyz". The "feature-" prefix
is used, because NMSettingEthtool possibly will gain support
for options that are not only -K|--offload|--features, for
example -C|--coalesce.
The "xzy" suffix is either how ethtool utility calls the feature
("tso", "rx"). Or, if ethtool utility specifies no alias for that
feature, it's the name from kernel's ETH_SS_FEATURES ("tx-tcp6-segmentation").
If possible, we prefer ethtool utility's naming.
Also note, how the features "feature-sg", "feature-tso", and
"feature-tx" actually refer to multiple underlying kernel features
at once. This too follows what ethtool utility does.
The functionality is not yet implemented server-side.
CC src/devices/ovs/src_devices_ovs_libnm_device_plugin_ovs_la-nm-device-ovs-bridge.lo
In file included from src/devices/ovs/nm-device-ovs-bridge.c:20:
In file included from ./shared/nm-default.h:307:
In file included from ./src/nm-logging.h:25:
./libnm-core/nm-core-types.h:28:10: fatal error: 'nm-core-enum-types.h' file not found
#include "nm-core-enum-types.h"
^~~~~~~~~~~~~~~~~~~~~~
1 error generated.
CC src/settings/plugins/ifupdown/src_settings_plugins_ifupdown_libnms_ifupdown_core_la-nms-ifupdown-interface-parser.lo
In file included from src/settings/plugins/ifupdown/nms-ifupdown-interface-parser.c:23:
In file included from ./shared/nm-default.h:307:
In file included from ./src/nm-logging.h:25:
./libnm-core/nm-core-types.h:28:10: fatal error: 'nm-core-enum-types.h' file not found
#include "nm-core-enum-types.h"
^~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [Makefile:13904: src/settings/plugins/ifupdown/src_settings_plugins_ifupdown_libnms_ifupdown_core_la-nms-ifupdown-interface-parser.lo] Error 1
Instead of letting each nmcli run write an individual .expected file,
combine the output of multiple runs in one file (per test).
Advantages:
- since there is a very large number of tests, having a file for each
tests is cumbersome. For example, since they are all added to
$(EXTRA_DIST) (and since we use non-recursive make), autoconf can
easily hit a length limit when processing all the file names.
- previously, whenever we added tests, all .expected files shifted
and the diff was large. Now, there is a chance that the diff is
smaller and more accurate.
We only have a certain granularity of how our headers in "shared/nm-utils"
can be used independently.
For example, it's not supported to use "nm-macros-internal.h" without
"gsystem-local-alloc.h". Likewise, you cannot use "nm-glib.h" directly,
you always get it together with "nm-macros-internal.h".
This is, we don't support to use certain headers entirely independently,
because usually you anyway want to use them together.
As such, no longer support "gsystem-local-alloc.h", but merge the
remainder into "nm-macros-internal.h". There is really no reason
to support arbitrary flexibility of including individual bits. You
want cleanup-macros? Include "nm-macros-internal.h".
Merge the headers.
With --enable-more-warnings, we already used -std=gnu99, see
commit ba2b2de3ad.
Compilation may behave differently depending on the selected
C standard that we choose. It seems wrong, with more-warnings,
to build against a C standard, while otherwise leaving it undefind.
Indeed, one might argue, that our build system should not use
such compiler specific options. At least, not without detecting
support for the compiler option during ./configure.
However:
- we already did this for --enable-more-warnings.
- we should not program against a theoretical compiler. In practice,
only gcc and clang works to build NetworkManager. Both these compilers
support this option, so there is no reason to not use it. If we ever
come into the situation to support another compiler, adjusting -std=gnu99
will be the smallest problem. Until that happens (and that's far from
imminent), don't pretend to be portable to non-existing compilers and
use the flag that in practice is available.
See-also: https://gcc.gnu.org/onlinedocs/gcc/Standards.html
1) the command line gets shorter. I frequently run `make V=1` to see
the command line arguments for the compiler, and there is a lot
of noise.
2) define each of these variables at one place. This makes it easy
to verify that for all compilation units, a particular
define has the same value. Previously that was not obvious or
even not the case (see commit e5d1a71396
and commit d63cf1ef2f).
The point is to avoid redundancy.
3) not all compilation units need all defines. In fact, most modules
would only need a few of these defines. We aimed to pass the necessary
minium of defines to each compilation unit, but that was non-obvious
to get right and often we set a define that wasn't used. See for example
"src_settings_plugins_ibft_cppflags" which needlessly had "-DSYSCONFDIR".
This question is now entirely avoided by just defining all variables in
a header. We don't care to find the minimum, because every component
gets anyway all defines from the header.
4) this also avoids the situation, where a module that previously did
not use a particular define gets modified to require it. Previously,
that would have required to identify the missing define, and add
it to the CFLAGS of the complation unit. Since every compilation
now includes "config-extra.h", all defines are available everywhere.
5) the fact that each define is now available in all compilation units
could be perceived as a downside. But it isn't, because these defines
should have a unique name and one specific value. Defining the same
name with different values, or refer to the same value by different
names is a bug, not a desirable feature. Since these defines should
be unique accross the entire tree, there is no problem in providing
them to every compilation unit.
6) the reason why we generate "config-extra.h" this way, instead of using
AC_DEFINE() in configure.ac, is due to the particular handling of
autoconf for directory variables. See [1].
With meson, it would be trivial to put them into "config.h.meson".
While that is not easy with autoconf, the "config-extra.h" workaround
seems still preferable to me.
[1] https://www.gnu.org/software/autoconf/manual/autoconf-2.63/html_node/Installation-Directory-Variables.html
It's for 6LoWPAN devices. "o.fd.NM.Device.6Lowpan" wouldn't be a valid
interface name -- just skip the leading numeral, that's what kernel also
does on similiar occassions.
They are known to be racy and occasionally break. Especially in
cases where the system's CPU is busy, like during parallel
`make check -j`.
It's likely a bug in libnm-glib. libnm-glib is deprecated, and the
library didn't significantly change now for several releases.
Let's not invest effort into finding bugs in the deprecated library,
bugs that are known to exist. Also, at this point, larger rework
of libnm-glib is not going to happen anymore.
Retry the test up to 5 times, trying to workaround the test failures.
They're intended to be used via update-alternatives(8) as compatibility
shims for Red Hat systems without the legacy network control scripts.
While they're not strictly parts of the settings plugin, they're best
just installed along with it, since they're supposed to be available on
systems that use the ifcfg files.
Originally, we used "nm-utils/siphash24.c", which was copied
from systemd's source tree. It was both used by our own NetworkManager
code, and by our internal systemd fork.
Then, we added "shared/c-siphash" as a dependency for n-acd.
Now, drop systemd's implementation and use c-siphash also
for our internal purpose. Also, let systemd code use c-siphash,
by patching "src/systemd/src/basic/siphash24.h".
Use two common defines NM_BUILD_SRCDIR and NM_BUILD_BUILDDIR
for specifying the location of srcdir and builddir.
Note that this is only relevant for tests, as they expect
a certain layout of the directories, to find files that concern
them.
All users are supposed to include files from nm-utils by fully specifying
the path. -I.*shared/nm-utils is wrong.
Only, systemd code likes to include "siphash24.h" directly. Instead of
adding "-Ishared/nm-utils" to the search path, add an intermediary
header to sd-adapt. Note, that in the meantime we anyway should rework
siphash24 to use shared/c-siphash instead.
This also fixes build for meson, which was broken recently.
The developer can re-generate .expected files with
$ NM_TEST_REGENERATE=1 ./clients/tests/test-client.py
Note that these files are also dist-ed, so that the tests also work
from a source-tarball. For that, we need to add them to EXTRA_DIST.
Previously, this was done manually in the base Makefile.am file. This
was cumbersome, because when adding a new test, the developer would need
to manually add the files.
Now, let the test (with NM_TEST_REGENERATE=1) also generate a makefile
part.
call_nmcli_l() would test for 3 languages: 'C', 'de', and 'pl'. There
is no fundamental difference between 'de' and 'pl', so there is no need
to test for two languages.
Activate the same profile on two devices. Arguably, real NetworkManager
(currently) does not allow that. But the D-Bus API is fine with
having multiple ActiveConnections for one SettingsConnection.
So, also the client should do something sensible.
Also, later we will add wildcard support to NetworkManager, which means
that a profile can be active multiple times (simultaneously).
Avoids an annoying warning in Fedora:
|DEPRECATION WARNING: python2 invoked with /usr/bin/python.
Use /usr/bin/python3 or /usr/bin/python2
/usr/bin/python will be removed or switched to Python 3 in the future.
Also allows the user to override the Python version that's actually
used.
This makes package updates more robust, avoiding in-place replaces of
the plugins.
Previously, if an upgrade transaction was terminated, NetworkManager
library could end up being of a different version than the plugins.
If the user was unfortunate enough to connect using a connection that
required a plugin (say, Wi-Fi), he would be left without a network
connection making it somewhat inconvenient to recover from the botched
upgrade.
This makes the whole situation a little bit less sad.
The VPN plugins are kept where they always have been -- the path is not
qualified with a version number.
Add a test which runs nmcli against our stub NetworkManager
service and compares the output.
The output formats of nmcli are complicated and not easily understood.
For example how --mode tabular|multiline interacts with selecting
output-fields (--fields) and output modes ([default]|--terse|--pretty).
Also, there are things like `nmcli connection show --order $FIELD_SPEC`.
We need unit tests to ensure that we don't change the output
accidentally.
When building with --disable-introspection, we re-use the pre-generated
clients/common/settings-doc.h.in file.
When building with --enable-introspection, we generate
clients/common/settings-doc.h, and let `make check` verify that the
generated file is identical to what we would generate.
The common case where the generated file differ, is when code changed,
in this case, the developer is advised to update settings-doc.h.in.
Interpret environment variable NM_TEST_REGENERATE=1 to do this
automatically during `make check`.
This will be useful, as there might be several tests that compare a
generated file with a file from version control. NM_TEST_REGENERATE=1
will be the general way to re-generate all these files.
$ NM_TEST_REGENERATE=1 make check
tools/test-networkmanager-service.py is our NetworkManager stub server.
NetworkManager uses libnm(-core) heavily, for example to decide whether
a connection verifies (nm_connection_verify()) and for normalizing
connections (nm_connection_normalize()).
If the stub server wants to mimic NetworkManager, it also must use these
function. Luckily, we already can do so, by loading libnm using python
GObject introspection.
We already correctly set GI_TYPELIB_PATH search path, so that the
correct libnm is loaded -- provided that we build with introspection
enabled.
We still need to gracefully fail, if starting the stub server fails.
That requries some extra effort. If the stub server notices that
something is missing, it shall exit with status 77. That will cause
the tests to g_test_skip().
The present version of the specification is somewhat unclear at times,
Unclear points were discussed with the maintainers [1] and probably
some new version will address those.
https://www.spinics.net/lists/util-linux-ng/msg15222.html
Until then here's how the implementation copes with ambiguities
(after the discussion with util-linux maintainers):
1.) It is unclear whether multiple .schem files should override each
other or be merged. We use the overriding behavior -- take the
highest priority one and ignore the rest.
2.) We assume "name.schem" is more specific than "@term.schem".
3.) We assume the "Color name" are to be used as aliases for the color
sequences and translate them to ANSI escape sequences.
4.) The "Escape sequences" are of no use since the specification
pretty much assumes an ANSI terminal and none of the sequences make
any sense in ANSI color codes. We don't support them.
accept that.
5.) We don't implement TERMINAL_COLORS_DEBUG because it's unspecified
what should it do.
This makes package updates more robust, avoiding in-place replaces of
the plugins.
Previously, if an upgrade transaction was terminated, NetworkManager
library could end up being of a different version than the plugins.
If the user was unfortunate enough to connect using a connection that
required a plugin (say, Wi-Fi), he would be left without a network
connection making it somewhat inconvenient to recover from the botched
upgrade.
This makes the whole situation a little bit less sad.
Load the thunderbolt-net module if we see a host-to-host connection
and configure the resulting ethernet connection automatically to be
a link-local only one. The latter is done by setting a new udev
property "NM_AUTO_DEFAULT_LINK_LOCAL_ONLY" which is picked up when
we configure the connection for the device.
https://github.com/NetworkManager/NetworkManager/pull/97
Splitting keyfile handling in two "reader.c" and "writer.c" files
is not helpful. What is most interesting, is to see how property XYZ
is serialized to keyfile, and to verify that the parser does the
inverse. For that, it's easier if both the write_xzy() and parse_xyz()
function are beside each other, and not split accross files.
The more important reason is, that both reader and writer have their
separate handler arrays, for special handling of certain properties:
@key_parsers and @key_writers. These two should not be separate but will
be merged. Since they reference static functions, these functions must
all be in the same source file (unless, we put them into headers, which
would be unnecessary complex).
No code was changed, only moved.
We don't need an external CheckpointItem, just to wrap the
CList instance. Embed it directly in NMCheckpoint.
Sure, that exposes the checkpoints_lst field in the (internal)
header file, hiding the private member less.
"nm-autoptr.h" is done in a way that allows you to copy the header
in your source tree to support older versions of libnm, that didn't
contain the header yet. For example, we might want to use it in
network-manager-applet, but we don't want to bump the libnm dependency
to 1.11.2+ only to get this functionality.
Note that G_DEFINE_AUTOPTR_CLEANUP_FUNC() was added in glib 2.43.4,
and requires compiler support for the cleanup attribute. The compiler
support is taken as given, because we rely on it already. However,
NetworkManager and network-manager-applet still don't depend on a glib
version recent enough to provide these macros. To actually use them
(*inside*) NetworkManager/network-manager-applet, we either would have
to bump the glib minimal dependency, or reimplement g_autoptr in
/shared/nm-utils/nm-glib.h compat header.
https://bugzilla.gnome.org/show_bug.cgi?id=794294
We already do conditional build with "#if WITH_CONCHECK".
Get rid of the conditional in the makefile and instead do
conditional compilating inside the source file "nm-connectivity.c".
The advantage is, now if you want to know which parts are build,
you only need to grep for the WITH_CONCHECK preprocessor define
instead of also caring about the conditional in Makefile.am and
meson.build.
It doesn't change the fact of conditional compilation. But it
consistently uses one mechanism to achieve it.
Previously, we used the generated GDBusInterfaceSkeleton types and glued
them via the NMExportedObject base class to our NM types. We also used
GDBusObjectManagerServer.
Don't do that anymore. The resulting code was more complicated despite (or
because?) using generated classes. It was hard to understand, complex, had
ordering-issues, and had a runtime and memory overhead.
This patch refactors this entirely and uses the lower layer API GDBusConnection
directly. It replaces the generated code, GDBusInterfaceSkeleton, and
GDBusObjectManagerServer. All this is now done by NMDbusObject and NMDBusManager
and static descriptor instances of type GDBusInterfaceInfo.
This adds a net plus of more then 1300 lines of hand written code. I claim
that this implementation is easier to understand. Note that previously we
also required extensive and complex glue code to bind our objects to the
generated skeleton objects. Instead, now glue our objects directly to
GDBusConnection. The result is more immediate and gets rid of layers of
code in between.
Now that the D-Bus glue us more under our control, we can address issus and
bottlenecks better, instead of adding code to bend the generated skeletons
to our needs.
Note that the current implementation now only supports one D-Bus connection.
That was effectively the case already, although there were places (and still are)
where the code pretends it could also support connections from a private socket.
We dropped private socket support mainly because it was unused, untested and
buggy, but also because GDBusObjectManagerServer could not export the same
objects on multiple connections. Now, it would be rather straight forward to
fix that and re-introduce ObjectManager on each private connection. But this
commit doesn't do that yet, and the new code intentionally supports only one
D-Bus connection.
Also, the D-Bus startup was simplified. There is no retry, either nm_dbus_manager_start()
succeeds, or it detects the initrd case. In the initrd case, bus manager never tries to
connect to D-Bus. Since the initrd scenario is not yet used/tested, this is good enough
for the moment. It could be easily extended later, for example with polling whether the
system bus appears (like was done previously). Also, restart of D-Bus daemon isn't
supported either -- just like before.
Note how NMDBusManager now implements the ObjectManager D-Bus interface
directly.
Also, this fixes race issues in the server, by no longer delaying
PropertiesChanged signals. NMExportedObject would collect changed
properties and send the signal out in idle_emit_properties_changed()
on idle. This messes up the ordering of change events w.r.t. other
signals and events on the bus. Note that not only NMExportedObject
messed up the ordering. Also the generated code would hook into
notify() and process change events in and idle handle, exhibiting the
same ordering issue too.
No longer do that. PropertiesChanged signals will be sent right away
by hooking into dispatch_properties_changed(). This means, changing
a property in quick succession will no longer be combined and is
guaranteed to emit signals for each individual state. Quite possibly
we emit now more PropertiesChanged signals then before.
However, we are now able to group a set of changes by using standard
g_object_freeze_notify()/g_object_thaw_notify(). We probably should
make more use of that.
Also, now that our signals are all handled in the right order, we
might find places where we still emit them in the wrong order. But that
is then due to the order in which our GObjects emit signals, not due
to an ill behavior of the D-Bus glue. Possibly we need to identify
such ordering issues and fix them.
Numbers (for contrib/rpm --without debug on x86_64):
- the patch changes the code size of NetworkManager by
- 2809360 bytes
+ 2537528 bytes (-9.7%)
- Runtime measurements are harder because there is a large variance
during testing. In other words, the numbers are not reproducible.
Currently, the implementation performs no caching of GVariants at all,
but it would be rather simple to add it, if that turns out to be
useful.
Anyway, without strong claim, it seems that the new form tends to
perform slightly better. That would be no surprise.
$ time (for i in {1..1000}; do nmcli >/dev/null || break; echo -n .; done)
- real 1m39.355s
+ real 1m37.432s
$ time (for i in {1..2000}; do busctl call org.freedesktop.NetworkManager /org/freedesktop org.freedesktop.DBus.ObjectManager GetManagedObjects > /dev/null || break; echo -n .; done)
- real 0m26.843s
+ real 0m25.281s
- Regarding RSS size, just looking at the processes in similar
conditions, doesn't give a large difference. On my system they
consume about 19MB RSS. It seems that the new version has a
slightly smaller RSS size.
- 19356 RSS
+ 18660 RSS
The next commit will completely rework NMBusManager and replace
NMExportedObject by a new type NMDBusObject.
Originally, NMDBusObject was added along NMExportedObject to ease
the rework and have compilable, intermediate stages of refactoring. Now,
I think the new name is better, because NMDBusObject is very strongly related
to the bus manager and the old name NMExportedObject didn't make that
clear.
I also slighly prefer the name NMDBusObject over NMBusObject, hence
for consistancy, also rename NMBusManager to NMDBusManager.
This commit only renames the file for a nicer diff in the next commit.
It does not actually update the type name in sources. That will be done
later.
From libnl3, we only used the helper function to parse/generate netlink
messages and the socket functions to send/receive messages. We don't
need an external dependency to do that, it is simple enough.
Drop the libnl3 dependency, and replace all missing code by directly
copying it from libnl3 sources. At this point, I mostly tried to
import the required bits to make it working with few modifications.
Note that this increases the binary size of NetworkManager by 4736 bytes
for contrib/rpm build on x86_64. In the future, we can simplify the code
further.
A few modifications from libnl3 are:
- netlink errors NLE_* are now in the domain or regular errno.
The distinction of having to bother with two kinds of error
number domains was annoying.
- parts of the callback handling is copied partially and unused parts
are dropped. Especially, the verbose/debug handlers are not used.
In following commits, the callback handling will be significantly
simplified.
- the complex handling of seleting ports was simplified. We now always
let kernel choose the right port automatically.
clang 5.0.1 complains:
src/systemd/src/libsystemd-network/dhcp6-option.c:40:28: error:
field 'option' with variable sized type 'struct DHCP6Option' not at
the end of a struct or class is a GNU extension
[-Werror,-Wgnu-variable-sized-type-not-at-end]
struct DHCP6Option option;
^
systemd disables this warning too.
Shared libraries built with sanitizers are a bit inconvenient to use
because they require that any application linking to them is run with
libasan preloaded using LD_PRELOAD. This limitation makes the
sanitizer support less useful because applications will refuse to
start unless there is a special environment variable set.
Let's turn the --enable-address-sanitizer configure flag into
--with-address-sanitizer=yes|no|exec so that is possible to enable
asan only for executables.
The ifcfg-rh plugin provides its own D-Bus service which initscripts
query to determine whether NetworkManager handles an ifcfg file.
Rework the D-Bus glue to hook GDBus with NetworkManager to use
GDBusConnection directly. Don't use generated code, don't use
GDBusInterfaceSkeleton.
We still keep "src/settings/plugins/ifcfg-rh/nm-ifcfg-rh.xml"
and still compile the static generated code. We don't actually need
them anymore, maybe the should be dropped later.
This is a proof of concept for reworking the D-Bus glue in
NetworkManager core to directly use GDBusConnection. Reworking core is
much more complicated, because there we also have properties, and a
class hierarchy.
Arguably, for the trivial ifcfg-rh service all this hardly matters, because
the entire D-Bus service only consists of one method, which is unlikely to
be extended in the future.
Now we get rid of layers of glue code, that were hard to comprehend.
Did you understand how nm_exported_object_skeleton_create() works
and uses the generated code and GDBusInterfaceSkeleton to hook into
GDBusConnection? Congratulations in that case. In my opinion, these
layers of code don't simplify but complicate the code.
The change also reduces the binary size of "libnm-settings-plugin-ifcfg-rh.so"
(build with contrib/rpm --without debug) by 8312 bytes (243672 vs. 235360).
We commonly only allow tabs at the beginning of a line, not
afterwards. The reason for this style is so that the code
looks formated right with tabstop=4 and tabstop=8.
Especially useful, because we don't link against libnl-genl-3.so
but re-implement generic netlink support. Such code should go there
so it can be used by various components.
The `settings-docs.c` file is generated by processing the
`nm-property-docs.xml` file. Although this works in autotools,
the `.c` extension makes meson not to handle it properly.
Given the fact that it only contains a number of defines it
makes sense to change its extension to `.h` an use it as a header.
This also makes meson to handle it properly and build it before
its used.
https://mail.gnome.org/archives/networkmanager-list/2018-January/msg00057.html
Although it is possible to generate distributable files on meson
since version 0.41 by using the `ninja dist` command, autotools does
different things that end up creating a different distributable
file.
meson build files have been added to autotools build files as
distributable files, so the whole meson port would also be
distributed.
https://mail.gnome.org/archives/networkmanager-list/2018-January/msg00047.html
Sometimes, we want to use CList to track a simple data item. But contrary
to GList/GSList, we need to define a structure to hold the data pointer
and the CList member.
Add a generic NMCListElem type that can be used for such simple uses.
Before you ask: why not use GList/GSList? Because even simple operations
like g_list_append() is O(n), which kinda defeats the purpose of having
a doubly linked list.
This code is added to a new header file nm-c-list.h, the reason is that
there is no other good place:
- "nm-utils/c-list.h" is a clone of upstream, it should not deviate.
- "nm-utils/c-list-util.h" contains our utils functions for c-list.h
but should be plain C, independent of glib.
- "nm-utils/nm-shared-utils.h" contains our glib related utilities,
but it should not drag in "c-list.h".
So, "nm-c-list.h" is a utility libray that extends "c-list.h" and
requires glib.
When doing changes that affect multiple source files, it's more
convenient to build the parts that have less dependencies first.
So, to fix the build failures from the core outward.
Note that:
- we compile some source files multiple times. Most notably those
under "shared/".
- we include a default header "shared/nm-default.h" in every source
file. This header is supposed to setup a common environment by defining
and including parts that are commonly used. As we always include the
same header, the header must behave differently depending
one whether the compilation is for libnm-core, NetworkManager or
libnm-glib. E.g. it must include <glib/gi18n.h> or <glib/gi18n-lib.h>
depending on whether we compile a library or an application.
For that, the source files need the NETWORKMANAGER_COMPILATION #define
to behave accordingly.
Extend the define to be composed of flags. These flags are all named
NM_NETWORKMANAGER_COMPILATION_WITH_*, they indicate which part of the
build are available. E.g. when building libnm-core.la itself, then
WITH_LIBNM_CORE, WITH_LIBNM_CORE_INTERNAL, and WITH_LIBNM_CORE_PRIVATE
are available. When building NetworkManager, WITH_LIBNM_CORE_PRIVATE
is not available but the internal parts are still accessible. When
building nmcli, only WITH_LIBNM_CORE (the public part) is available.
This granularily controls the build.
Even Gentoo disables this plugin since before 0.9.8 release
of NetworkManager. Time to say goodbye.
If somebody happens to show up to maintain it, we may resurrect it
later.
If "$distro_plugins=ifnet" was set, configure.ac would use that
to autodetect --with-hostname-persist=gentoo. Replace that autodetect
part by checking for /etc/gentoo-release file.
There are three headers `nm-secret-agent-old.h`,
`nm-vpn-plugin-old.h`, and `nm-vpn-service-plugin.h`, which are
named as no introspection headers. However, these files also
join to the rest headers to generate introspection data.
This patch merges those no introspection headers with the public
headers.
`generate-plugin-docs.pl` script which is used to parse
`nm-setting-c*.c` files depends on autotools. This is because it
parses the `Makefile.am` in order to figure out the setting files
it needs to parse.
This patch makes the script independent of autotools by passing
the necessary setting files by command line instead of parsing the
`Makefile.am` file. It also changes the autotools' and meson's
accordingly.
Add the NMIwdManager singleton to be responsible for matching
NMDeviceIwd objects created from platform devices, to IWD Device dbus
objects when they appear/disappear.
This is very similar to NMDeviceWifi but simplified to remove the things
currently unsupported and with calls to nm_platform_wifi_* and
nm_supplicant_* replaced with IWD DBus API calls. Only unsecured
infrastructure-mode networks are supported here.
[bgalvani@redhat.com: fix compilation error after rebase for
NMActRequestGetSecretsCallId]
[thaller@redhat.com: don't use _() macro strings server side.
Translating strings only makes sense for clients that set environment
variables accordingly.]
Systemd instroduces a macro _fallthrough_, see
https://github.com/systemd/systemd/pull/7389.
However, it does not yet seem conclusive how to
handle this properly in ever situation.
While shared/nm-utils/siphash24.c makes use of
the new macro, don't do that in our fork. siphash24.h
does not include all systemd headers, hence _fallthrough_
is not defined. We could re-implement it as _nm_fallthrough,
but given the open questions, that doesn't seem the
Also, don't add the CFLAGS for libjansson to dflt_cppflags_libnm_core.
dflt_cppflags_libnm_core is used also by core and libnm. But those
components do not (directly) link against libjansson. The cannot use
these flags.
I don't think we should do this.
- renamining/dropping configure options is still an annoyance,
because it requires to different ./configure options depending
on the version. The rename from --enable-teamctl to --enable-team
might be theoretically nice, but more annoying then helpful.
- There is no strict dependency between --enable-team and
--enable-json-validation. At most, one could argue that
when enabling the team plugin (--enable-teamctl), then
libnm must also be build with --enable-json-validation.
But in fact, the team plugin will happily work with a
libnm that doesn't link against libjansson.
That is --enable-teamctl --disable-json-validation will work
in practice just fine.
On the other hand, libnm is a client library to create connection
profiles, fully supporting team profiles also makes sense if the
actual plugin is not installed (or build). Thus, --disable-teamctl
--enable-json-validation certainly makes sense.
At this point, one might ask whether libnm is even still complete without
libjansson. Maybe libnm should *require* --enable-json-validation.
But that is not what the patch was doing, and it would also need
some careful consideration before doing so.
This reverts commit 9d5cd7eae8.
Rename the team functionality enablement from 'teamdctl' to 'team'.
Force jansson lib requirement for team functionality: NetworkManager
requires the teamd daemon to manage team. As teamd depends upon jansson
lib, adding jansson requirement for teaming support in NetworkManager
seems reasonable.
Remove the jansson_validation flag, as the only generic json function in
nmcli (not related to team) was the one to check if a string was in json
format. Anyway, that function is used for team checks only. So, move
also json validation functions under the WITH_TEAM flag.
This allows the compiler to inline the siphash24*() functions
for nm_hash_ptr() and nm_hash_str() (even without LTO).
This of course only applies to nm_hash_ptr() and nm_hash_str(),
which are implemented in "nm-hash-utils.c" itself. All other
nm_hash_*() functions are inline functions in "nm-hash-utils.h",
and thus these functions can be inlined instead. That is, in
other cases, the nm_hash_*() function instead can be inlined.
For nm_hash_ptr() and nm_hash_str() instead we want to inline the
siphash24*() functions.
So, no longer compile "siphash24.c" directly. Instead, only
build "nm-hash-utils.c" which internally #include "siphash24.c".
CC src/systemd/src/basic/src_libsystemd_nm_la-string-table.lo
../../src/systemd/src/basic/parse-util.c:30:10: fatal error: 'errno-list.h' file not found
#include "errno-list.h"
^~~~~~~~~~~~~~
Fixes: 1a2419a0c9
Kernel doesn't support it for IPv6.
This is especially useful, if you combine static routes
with DHCP. In that case, you might want to get the device-route
to the gateway automatically, but add a static-route for it.
Fixes the following errors in 'make distcheck':
1)
GEN libnm/fake-typelib/NMClient.typelib
failed to open 'libnm/fake-typelib/NMClient.typelib.tmp': No such file or directory
make[2]: *** [libnm/fake-typelib/NMClient.typelib] Error 1
2)
GEN libnm/fake-typelib/typelibs.c
../../libnm/fake-typelib/typelibs.gresource.xml: Failed to locate “NetworkManager.typelib” in any source directory.
make[1]: *** [Makefile:17790: libnm/fake-typelib/typelibs.c] Error 1
3)
ERROR: files left in build directory after distclean:
./libnm/fake-typelib/typelibs.c
./libnm/fake-typelib/NMClient.typelib
./libnm/fake-typelib/NetworkManager.typelib
Fixes: 4d1f090aed
Currently the ifcfg-rh plugin doesn't explicitly store the connection
type for team slaves and is only able to read back ethernet and vlan
connections.
Leave this unchanged for ethernet and vlan slaves, but store the TYPE
variable for other connection types (Wi-Fi and Infiniband) so that we
can properly determine their type when the connection is read.
Travis' version of glib-compile-resources does not support the
--internal flag:
GEN libnm/fake-typelib/typelibs.c
Unknown option --internal
make: *** [libnm/fake-typelib/typelibs.c] Error 1
We don't really need it anyway, because we have a linker version
script, that controls symbol visibility.
Fixes: 4d1f090aed
Register empty "NMClient" and "NetworkManager" GIR modules as soon as libnm is
loaded witch gnome-introspection. This prevents the real modules from being
loaded because they would in turn load libnm-glib and abort() and crash.
In particular this prevents the GNOME shell from crashing with
libnm-glib abort and allows gracefully disabling the extensions which
use the obsolete library.
Test:
$ cat test.js
const NM = imports.gi.NM;
print (NM.SecretAgentGetSecretsFlags.ALLOW_INTERACTION);
const NMClient = imports.gi.NMClient;
print (NMClient.SecretAgentGetSecretsFlags.ALLOW_INTERACTION);
Before:
$ gjs test.js
1
(gjs:16253): libnm-util-ERROR **: libnm symbols detected; Mixing libnm with libnm-util/libnm-glib is not supported
Trace/breakpoint trap (core dumped)
$
After:
$ gjs test.js
1
Gjs-Message: JS WARNING: [test.js 5]: reference to undefined property "SecretAgentGetSecretsFlags"
(gjs:16228): Gjs-WARNING **: JS ERROR: TypeError: NMClient.SecretAgentGetSecretsFlags is undefined
@test.js:5:1
JS_EvaluateScript() failed
When the ifcfg-rh plugin writes a 802-1x setting it currently ignores
the password-raw property and so the password disappears when the
connection is saved. Add support for the property.
