We have "nm-logging-fwd.h", which (as the name implies) is header-only.
Add instead a "nm-logging-base.c", which also contains implementation for
logging functions that are not only useful under "src/nm-logging.c"
No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes
for the NMClient cache. Instead, use GDBusConnection directly and a
custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager
data.
CHANGES
-------
- This is a complete rework. I think the previous implementation was
difficult to understand. There were unfixed bugs and nobody understood
the code well enough to fix them. Maybe somebody out there understood the
code, but I certainly did not. At least nobody provided patches to fix those
issues. I do believe that this implementation is more straightforward and
easier to understand. It removes a lot of layers of code. Whether this claim
of simplicity is true, each reader must decide for himself/herself. Note
that it is still fairly complex.
- There was a lingering performance issue with large number of D-Bus
objects. The patch tries hard that the implementation scales well. Of
course, when we cache N objects that have N-to-M references to other,
we still are fundamentally O(N*M) for runtime and memory consumption (with
M being the number of references between objects). But each part should behave
efficiently and well.
- Play well with GMainContext. libnm code (NMClient) is generally not
thread safe. However, it should work to use multiple instances in
parallel, as long as each access to a NMClient is through the caller's
GMainContext. This follows glib's style and effectively allows to use NMClient
in a multi threaded scenario. This implies to stick to a main context
upon construction and ensure that callbacks are only invoked when
iterating that context. Also, NMClient itself shall never iterate the
caller's context. This also means, libnm must never use g_idle_add() or
g_timeout_add(), as those enqueue sources in the g_main_context_default()
context.
- Get ordering of messages right. All events are consistently enqueued
in a GMainContext and processed strictly in order. For example,
previously "nm-object.c" tried to combine signals and emit them on an
idle handler. That is wrong, signals must be emitted in the right order
and when they happen. Note that when using GInitable's synchronous initialization
to initialize the NMClient instance, NMClient internally still operates fully
asynchronously. In that case NMClient has an internal main context.
- NMClient takes over most of the functionality. When using D-Bus'
ObjectManager interface, one needs to handle basically the entire state
of the D-Bus interface. That cannot be separated well into distinct
parts, and even if you try, you just end up having closely related code
in different source files. Spreading related code does not make it
easier to understand, on the contrary. That means, NMClient is
inherently complex as it contains most of the logic. I think that is
not avoidable, but it's not as bad as it sounds.
- NMClient processes D-Bus messages and state changes in separate steps.
First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and
keeps track of the changed data. Then we update the GObject instances
(_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally,
we emit all signals and notifications that were collected
(_dbus_handle_changes_commit()). Note that for example during the initial
GetManagedObjects() reply, NMClient receive a large amount of state at once.
But we first apply all the changes to our GObject instances before
emitting any signals. The result is that signals are always emitted in a moment
when the cache is consistent. The unavoidable downside is that when you receive
a property changed signal, possibly many other properties changed
already and more signals are about to be emitted.
- NMDeviceWifi no longer modifies the content of the cache from client side
during poke_wireless_devices_with_rf_status(). The content of the cache
should be determined by D-Bus alone and follow what NetworkManager
service exposes. Local modifications should be avoided.
- This aims to bring no API/ABI change, though it does of course bring
various subtle changes in behavior. Those should be all for the better, but the
goal is not to break any existing clients. This does change internal
(albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER
property and NMObject no longer implementing GInitableIface and GAsyncInitableIface.
- Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin
and NMSecretAgentOld. These are independent of NMClient/NMObject and
should be reworked separately.
- While we no longer use generated classes from gdbus-codegen, we don't
need more glue code than before. Also before we constructed NMPropertiesInfo and
a had large amount of code to propagate properties from NMDBus* to NMObject.
That got completely reworked, but did not fundamentally change. You still need
about the same effort to create the NMLDBusMetaIface. Not using
generated bindings did not make anything worse (which tells about the
usefulness of generated code, at least in the way it was used).
- NMLDBusMetaIface and other meta data is static and immutable. This
avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U()
have compile time checks to ensure the property types matches. It's pretty hard
to misuse them because it won't compile.
- The meta data now explicitly encodes the expected D-Bus types and
makes sure never to accept wrong data. That would only matter when the
server (accidentally or intentionally) exposes unexpected types on
D-Bus. I don't think that was previously ensured in all cases.
For example, demarshal_generic() only cared about the GObject property
type, it didn't know the expected D-Bus type.
- Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those
probably indicated real bugs. In any case, it prevented us from running CI
with G_DEBUG=fatal-warnings, because there would be just too many
unrelated crashes. Now we log debug messages that can be enabled with
"LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned
into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error.
Together with G_DEBUG=fatal-warnings, this turns them into assertions.
Note that such "assertion failures" might also happen because of a server
bug (or change). Thus these are not common assertions that indicate a bug
in libnm and are thus not armed unless explicitly requested. In our CI we
should now always run with LIBNM_CLIENT_DEBUG=warning,error and
G_DEBUG=fatal-warnings and to catch bugs. Note that currently
NetworkManager has bugs in this regard, so enabling this will result in
assertion failures. That should be fixed first.
- Note that this changes the order in which we emit "notify:devices" and
"device-added" signals. I think it makes the most sense to emit first
"device-removed", then "notify:devices", and finally "device-added"
signals.
This changes behavior for commit 52ae28f6e5 ('libnm: queue
added/removed signals and suppress uninitialized notifications'),
but I don't think that users should actually rely on the order. Still,
the new order makes the most sense to me.
- In NetworkManager, profiles can be invisible to the user by setting
"connection.permissions". Such profiles would be hidden by NMClient's
nm_client_get_connections() and their "connection-added"/"connection-removed"
signals.
Note that NMActiveConnection's nm_active_connection_get_connection()
and NMDevice's nm_device_get_available_connections() still exposes such
hidden NMRemoteConnection instances. This behavior was preserved.
NUMBERS
-------
I compared 3 versions of libnm.
[1] 962297f908, current tip of nm-1-20 branch
[2] 4fad8c7c64, current master, immediate parent of this patch
[3] this patch
All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31.
The libraries were build with
$ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug
Note that RPM build already stripped the library.
---
N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue
on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but
in these tests it had large (and bogus) ELF notes. Anyway, the point
is to show the relative sizes, so it doesn't matter).
[1] 4075552 (102.7%)
[2] 3969624 (100.0%)
[3] 3705208 ( 93.3%)
---
N2) `size /usr/lib64/libnm.so.0.1.0`:
text data bss dec hex filename
[1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0
[2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0
[3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0
---
N3) Performance test with test-client.py. With checkout of [2], run
```
prepare_checkout() {
rm -rf /tmp/nm-test && \
git checkout -B test 4fad8c7c64 && \
git clean -fdx && \
./autogen.sh --prefix=/tmp/nm-test && \
make -j 5 install && \
make -j 5 check-local-clients-tests-test-client
}
prepare_test() {
NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v
}
do_test() {
for i in {1..10}; do
NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1
done
echo "done!"
}
prepare_checkout
prepare_test
time do_test
```
[1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%)
[2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%)
[3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%)
---
N4) Performance. Run NetworkManager from build [2] and setup a large number
of profiles (551 profiles and 515 devices, mostly unrealized). This
setup is already at the edge of what NetworkManager currently can
handle. Of course, that is a different issue. Here we just check how
long plain `nmcli` takes on the system.
```
do_cleanup() {
for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do
nmcli connection delete uuid "$UUID"
done
for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do
nmcli device delete "$DEVICE"
done
}
do_setup() {
do_cleanup
for i in {1..30}; do
nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore
for j in $(seq $i 30); do
nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore
done
done
systemctl restart NetworkManager.service
sleep 5
}
do_test() {
perf stat -r 50 -B nmcli 1>/dev/null
}
do_test
```
[1]
Performance counter stats for 'nmcli' (50 runs):
456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% )
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
5,900 page-faults:u # 0.013 M/sec ( +- 0.02% )
1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% )
1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% )
368,744,018 branches:u # 808.061 M/sec ( +- 0.02% )
4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% )
0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% )
[2]
Performance counter stats for 'nmcli' (50 runs):
477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% )
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
5,948 page-faults:u # 0.012 M/sec ( +- 0.03% )
1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% )
1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% )
382,595,152 branches:u # 800.433 M/sec ( +- 0.02% )
4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% )
0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% )
[3]
Performance counter stats for 'nmcli' (50 runs):
352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% )
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
4,790 page-faults:u # 0.014 M/sec ( +- 0.26% )
1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% )
1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% )
281,708,462 branches:u # 799.499 M/sec ( +- 0.01% )
3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% )
0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% )
---
N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`:
[1]
Command being timed: "nmcli"
User time (seconds): 0.42
System time (seconds): 0.04
Percent of CPU this job got: 107%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 34456
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 6128
Voluntary context switches: 1298
Involuntary context switches: 1106
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
[2]
Command being timed: "nmcli"
User time (seconds): 0.44
System time (seconds): 0.04
Percent of CPU this job got: 108%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 34452
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 6169
Voluntary context switches: 1849
Involuntary context switches: 142
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
[3]
Command being timed: "nmcli"
User time (seconds): 0.32
System time (seconds): 0.02
Percent of CPU this job got: 102%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 29196
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 5059
Voluntary context switches: 919
Involuntary context switches: 685
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
---
N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for
the RSS size.
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
[1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor
[2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor
[3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
We will rework NMClient entirely. Then, the synchronous initialization will also use
the asynchronous code paths. The difference will be that with synchronous initialization,
all D-Bus interaction will be done with an internal GMainContext as current thread default,
and that internal context will run until initialization completes.
Note that even after initialization completes, it cannot be swapped back to the user's
(outer) GMainContext. That is because contexts are essentially the queue for our
D-Bus events, and we cannot swap from one queue to the other in a race
free manner (or a full resync). In other words, the two contexts are not in sync,
so after using the internal context NMClient needs to stick to that (at least, until
the name owner gets lost, which gives an opportunity to resync and switch back to the
user's main context).
We thus need to hook the internal (inner) GMainContext with the user's (outer) context,
so when the user iterates the outer context, events on the inner context get dispatched.
Add nm_utils_g_main_context_create_integrate_source() to create such a GSource for
integrating two contexts.
Note that the use-case here is limited: the integrated, inner main context must
not be explicitly iterated except from being dispatched by the integrating
source. Otherwise, you'd get recursive runs, possible deadlocks and general
ugliness. NMClient must show restrain how to use the inner context while it is
integrated.
Some compilers don't convert arrays as _Generic() type selectors to
their pointer type. That means, for those compilers the generic type
would be an array and not a pointer. Work around that by adding zero
to the pointer/array argument.
Also, I cannot get this to work with "clang-3.4.2-9.el7". Disable it
for that compiler. The value of the generic check is anyway that it only
needs to work with some compiler combinations. That will trigger a
compilation failure and we can fix the implementation also for compilers
that don't support the macro.
See-also: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1930.htm
There are two macros: NM_GOBJECT_PROPERTIES_DEFINE_BASE() and
NM_GOBJECT_PROPERTIES_DEFINE(). The former just defines the
property enums and the obj_properties array. The latter also
defines the functions _notify() and _nm_gobject_notify_together_impl().
That means, depending on whether you actually use _notify(), you have
to choose one of the macros. I think that is unnecessarily cumbersome.
Let's mark the function as _nm_unused so that the compiler doesn't
complain about the unused function. I don't think it's a problem
to use NM_GOBJECT_PROPERTIES_DEFINE() even if you don't actually use
_notify().
We don't need such data duplicated. The build setup should
have only one configuration_data() for patching such values.
Now we only have one global, immutable data_conf dictionary with
configuration values. Note that none of the users of data_conf uses all
entries, but as the entries are basically only dependent on the
meson/configure option and valid for the entire project, this simplifies
to handling.
We preferably should use our convenience macros like _LOGD().
Since those macros expand to _NMLOG() (which needs to be defined
separately), we can move it to "nm-logging-fwd.h" and reuse.
glib really likes the numeric source IDs. That is, g_idle_add(), g_timeout_add(),
etc. return those IDs, that can then be destroyed with g_remove_source() (or
nm_clear_g_source()).
I think these numeric IDs are really not great.
- API like g_idle_add() and g_remove_source() only works with the g_main_context_get_default()
instance. That means, you cannot use this API for any other contexts. If you'd insist on using
numeric IDs, you'd need to call g_main_context_find_source_by_id() on the right context
first (but you'd also have to track the context alongside the ID).
- g_remove_source() requires first a call to g_main_context_find_source_by_id(). This involves
taking a mutex and doing an extra hash lookup.
Instead, it often seems preferable to use the GSource instance directly. It works
with any context, it can be referenced and unreferenced, and it can be destroyed, and
avoids the overhead of g_main_context_find_source_by_id().
The only downside really is that keeping a GSource pointer takes one pointer size, while
the guint source ID is usually only 4 bytes.
Anyway, I think we should deal more with GSource instances directly. Hence, add this
convenience macro, that works like nm_clear_g_source().
In some cases it is useful to have the library log what it is doing
for debugging purposes; add a simple API that allows setting a
syslog-style logging level and specifying a logging function.
https://github.com/nettools/n-dhcp4/pull/8
Currently in any of the BOUND, RENEWING and REBINDING states the probe
checks the expiration of T1, T2 and lifetime. This is not correct
because, for example, if the timer fires in the RENEWING state, the
probe must not transition to RENEWING again (i.e. check again that
now >= T1). Note that there is no guarantee that the timer triggers
exactly once for T1, T2 and lifetime expirations because the timer is
also used for the retransmission logic in NDhcp4CConnection.
Therefore, add some checks to ensure that only correct transitions are
allowed.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/341https://bugzilla.redhat.com/show_bug.cgi?id=1773456
The API already had n_dhcp4_client_lease_get_lifetime(), which is the CLOCK_BOOTTIME
when the lease expires (or ((uint64_t)-1)). But it might be interesting to
know the actual lease duration and when the lease was received (and the
time started to count).
Expose an API for that. With this, one can also calculate the original, exact lease
lifetime, by subtracting n_dhcp4_client_lease_get_basetime() from n_dhcp4_client_lease_get_lifetime(),
while taking care of ((uint64_t)-1).
First of all, from the naming of n_dhcp4_incoming_query_u32() it is
confusing to coerce 0xFFFFFFFF to zero. It should just return the
plain value.
Also note that n_dhcp4_incoming_query_u32() only has three callers:
n_dhcp4_incoming_query_lifetime(), n_dhcp4_incoming_query_t1() and
n_dhcp4_incoming_query_t2().
Looking further, those three functions only have one caller:
n_dhcp4_incoming_get_timeouts(). Note how the code there already tries
to handle UINT32_MAX and interprets it as infinity (UINT64_MAX).
But as it was, UINT32_MAX never actually was returned.
It seems that RFC [1] does not specially define the meanings of
0xFFFFFFFF and 0. It sounds reasonable to assume that 0 just means
0 lifetime, and 0xFFFFFFFF means infinity. On the other hand, compare
this to systemd's code [2], which coerces 0 to 1. This does not seem
right to me though. Note how systemd returns 0xFFFFFFFF as-is.
Drop the special handling of 0xFFFFFFFF from n_dhcp4_incoming_query_u32().
It now just returns the plain value and it's up to n_dhcp4_incoming_get_timeouts()
to make sense of that. This will fix behavior, so that 0xFFFFFFFF will be
reported as infinity, and not as zero.
[1] https://tools.ietf.org/html/rfc2132#section-9.2
[2] 68c2b5ddb1/src/libsystemd-network/sd-dhcp-lease.c (L553)
It is error prone when a function consumes an input only in certain
cases (and telling the caller via the return code). At least in these
cases, the message is never used afterwards, and we can always pass
it on.
The majority of sources in "libnm/" are implementations of NMObject.
"nm-libnm-utils.h" will contain common definitions for handling such
objects. This means, most of the source files under libnm will require
this include. Include it by default.
Note that we should always set the source-tag of our GTask.
This allows us to better assert that the user uses the right
_finish() method for the task.
The plain g_task_new() does not have a souce-tag argument. Hence, we would
always need to explicitly call g_task_set_source_tag().
Likewise, to check the source tag, we would always need to write
g_return_val_if_fail (g_task_is_valid (result, self), FALSE);
g_return_val_if_fail (g_async_result_is_tagged (result, tag), FALSE);
Actually, g_async_result_is_tagged() uses the GAsyncResultIface to
call iface->is_tagged(). This has unnecessary overhead, so we should
just call g_task_get_source_tag() directly.
Add helper functions for that.
The `libnm-core` build file has been improved by applying a set of
changes:
- Indentation has been fixed to be consistent.
- Library variable names have been changed to `lib{name}` pattern
following their filename pattern.
- `shared` prefix has been removed from all variables using it.
- Dependencies have been reviewed to store the necessary data.
- The use of the libraries and dependencies created in this file
has been reviewed through the entire source code. This has
required the addition or the removal of different libraries and
dependencies in different targets.
- Some files used directly with the `files` function have been moved
to their nearest path build file because meson stores their full
path seamessly and they can be used anywhere later.
The `nm-default.h` header is used widely in the code by many
targets. This header includes different headers and needs different
libraries depending the compilation flags.
A new set of `*nm_default_dep` dependencies have been created to
ease the inclusion of different directorires and libraries.
This allows cleaner build files and avoiding linking unnecessary
libraries so this has been applied allowing the removal of some
dependencies involving the linking of unnecessary libraries.
The `shared` build file has been improved by applying a set of
changes:
- Indentation has been fixed to be consistent.
- Unused libraries and dependencies have been removed.
- Dependencies have been reviewed to store the necessary data.
- Set of objects used in targets have been grouped together.
- Header files have been removed from sources lists as it's
unnecessary.
- Library variable names have been changed to `lib{name}` pattern
following their filename pattern.
- `shared` prefix has been removed from all variables using it.
- `version_header` its related configuration `version_conf`
variables have been renamed to `nm_version_macro*` following
its input and final file names.
Functions derived from generators as `configure_file`,
`custom_target` and `i18n.merge_file` can use placeholders like
`@BASENAME@` that removes the extension from the input filename
string.
The output string has been replaced by this placeholder that
allows in some cases the use of less variables.
Several points.
- We spawn the dnsmasq process directly. That has several downsides:
- The lifetime of the process is tied to NetworkManager's. When
stopping NetworkManager, we usually also stop dnsmasq. Or we keep
the process running, but later the process is no longer a child process
of NetworkManager and we can only kill it using the pidfile.
- We don't do special sandboxing of the dnsmasq process.
- Note that we want to ensure that only one dnsmasq process is running
at any time. We should track that in a singletone. Note that NMDnsDnsmasq
is not a singleton. While there is only one instance active at any time,
the DNS plugin can be swapped (e.g. during SIGHUP). Hence, don't track the
process per-NMDnsDnsmasq instance, but in a global variable "gl_pid".
- Usually, when NetworkManager quits, it also stops the dnsmasq process.
Previously, we would always try to terminate the process based on the
pidfile. That is wrong. Most of the time, NetworkManager spawned the
process itself, as a child process. Hence, the PID is known and NetworkManager
will get a signal when dnsmasq exits. The only moment when NetworkManager should
use the pidfile, is the first time when checking to kill the previous instance.
That is: only once at the beginning, to kill instances that were
intentionally or unintentionally (crash) left running earlier.
This is now done by _gl_pid_kill_external().
- Previously, before starting a new dnsmasq instance we would kill a
possibly already running one, and block while waiting for the process to
disappear. We should never block. Especially, since we afterwards start
the process also in non-blocking way, there is no reason to kill the
existing process in a blocking way. For the most part, starting dnsmasq
is already asynchronous and so should be the killing of the dnsmasq
process.
- Drop GDBusProxy and only use GDBusConnection. It fully suffices.
- When we kill a dnsmasq instance, we actually don't have to wait at
all. That can happen fully in background. The only pecularity is that
when we restart a new instance before the previous instance is killed,
then we must wait for the previous process to terminate first. Also, if
we are about to exit while killing the dnsmasq instance, we must register
nm_shutdown_wait_obj_*() to wait until the process is fully gone.
Having G_PID_FORMAT macro is useful, but it's only available in
recent glib versions. Add a compat implementation and a test that
our assumptions hold.
The leading underscore has the notion that this would be a private function.
It really isn't, and it would be fine for the user to call it directly.
Just like we have g_slice_free() and g_slice_free1().
I'd like to refactor libnm's caching. Note that cached D-Bus objects
have repeated strings all over the place. For example every object will
have a set of D-Bus interfaces (strings) and properties (strings) and an
object path (which is referenced by other objects). We can save a lot of
redundant strings by deduplicating/interning them. Also, by interning
them, we can compare them using pointer equality.
Add a NMRefString implementation for this.
Maybe an alternative name would be NMInternedString or NMDedupString, because
this string gets always interned. There is no way to create a NMRefString
that is not interned. Still, NMRefString name sounds better. It is ref-counted
after all.
Notes:
- glib has GQuark and g_intern_string(). However, such strings cannot
be unrefered and are leaked indefinitely. It is thus unsuited for
anything but a fixed set of well-known strings.
- glib 2.58 adds GRefString, but we cannot use that because we
currently still use glib 2.40.
There are some differences:
- GRefString is just a typedef to char. That means, the glib API
exposes GRefString like regular character strings.
NMRefString intentionally does that not. This makes it slightly
less convenient to pass it to API that expects "const char *".
But it makes it clear to the reader, that an instance is in fact
a NMRefString, which means it indicates that the string is
interned and can be referenced without additional copy.
- GRefString can be optionally interned. That means you can
only use pointer equality for comparing values if you know
that the GRefString was created with g_ref_string_new_intern().
So, GRefString looks like a "const char *" pointer and even if
you know it's a GRefString, you might not know whether it is
interned. NMRefString is always interned, and you can always
compare it using pointer equality.
- In the past I already proposed a different implementation for a
ref-string. That made different choices. For example NMRefString
then was a typedef to "const char *", it did not support interning
but deduplication (without a global cache), ref/unref was not
thread safe (but then there was no global cache so that two threads
could still use the API independently).
The point is, there are various choices to make. GRefString, the
previous NMRefString implementation and the one here, all have pros and
cons. I think for the purpose where I intend NMRefString (dedup and
efficient comparison), it is a preferable implementation.
Ah, and of course NMRefString is an immutable string, which is a nice
property.
There is really no reason for this to be a macro. Our hash-related
helpers (like nm_hash_update_val()) are macros because they do some
shenigans to accept arguments of different (compile-time) types. But
the arguments for nm_hash_obfuscate_ptr() are well known and expected
of a certain form.
Note that with "-O2" some quick testing shows that the compiler no
longer inlines the function. But I guess that's fine, probably the
compiler knows best anyway.
... and nm_utils_fd_get_contents() and nm_utils_file_set_contents().
Don't mix negative errno return value with a GError output. Instead,
return a boolean result indicating success or failure.
Also, optionally
- output GError
- set out_errsv to the positive errno (or 0 on success)
Obviously, the return value and the output arguments (contents, length,
out_errsv, error) must all agree in their success/failure result.
That means, you may check any of the return value, out_errsv, error, and
contents to reliably detect failure or success.
Also note that out_errsv gives the positive(!) errno. But you probably
shouldn't care about the distinction and use nm_errno_native() either
way to normalize the value.
nm_utils_file_set_contents() is a re-implementation of g_file_set_contents(),
as such it returned merely a boolean success value.
It's sometimes interesting to get the native error code. Let the function
deviate from glib's original g_file_set_contents() and return the error code
(as negative value) instead.
This requires all callers to change. Also, it's potentially a dangerous
change, as this is easy to miss.
Note that nm_utils_file_get_contents() also returns an errno, and
already deviates from g_file_get_contents() in the same way. This patch
resolves at least the inconsistency with nm_utils_file_get_contents().
- add nm_c_list_elem_find_first() macro that takes a predicate
and returns the first match.
This macro has a non-function-like behavior, which we often try to
avoid because macros should behave like functions. In this case it's
however convenient, so let's do it.
Also, despite being non-function-like, it should be pretty hard to
use wrongly.
- rename nm_c_list_elem_find_first() to nm_c_list_elem_find_first_ptr().
When we build n-dhcp4 for NetworkManager we get a compiler warning.
This can also be reproduced by building n-dhcp4 alone:
$ CFLAGS='-Werror=declaration-after-statement' meson build && ninja -C build
...
[36/47] Compiling C object 'src/25a6634@@ndhcp4-private@sta/n-dhcp4-outgoing.c.o'.
FAILED: src/25a6634@@ndhcp4-private@sta/n-dhcp4-outgoing.c.o
ccache cc -Isrc/25a6634@@ndhcp4-private@sta -Isrc -I../src -Isubprojects/c-list/src -I../subprojects/c-list/src -Isubprojects/c-siphash/src -I../subprojects/c-siphash/src -Isubprojects/c-stdaux/src -I../subprojects/c-stdaux/src -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c11 -g -D_GNU_SOURCE -Werror=declaration-after-statement -fPIC -fvisibility=hidden -fno-common -MD -MQ 'src/25a6634@@ndhcp4-private@sta/n-dhcp4-outgoing.c.o' -MF 'src/25a6634@@ndhcp4-private@sta/n-dhcp4-outgoing.c.o.d' -o 'src/25a6634@@ndhcp4-private@sta/n-dhcp4-outgoing.c.o' -c ../src/n-dhcp4-outgoing.c
../src/n-dhcp4-outgoing.c: In function ‘n_dhcp4_outgoing_new’:
../src/n-dhcp4-outgoing.c:63:9: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
63 | static_assert(N_DHCP4_NETWORK_IP_MINIMUM_MAX_SIZE >= N_DHCP4_OUTGOING_MAX_PHDR +
| ^~~~~~~~~~~~~
Coverity says
CID 202453 (#1 of 1): Wrong sizeof argument (SIZEOF_MISMATCH)suspicious_sizeof:
Passing argument user_data of type gconstpointer and argument (gsize)nargs * 8UL /* sizeof (gconstpointer) */ to function g_slice_free1 is suspicious.
Let's pass instead the "data" pointer. It's the same, but maybe that
avoids the warning.
Coverity doesn't like us ignoring the return value, although
we really only care about the "p" output pointer.
Try casting the result to (void), maybe that silences Coverity.
- don't let no_auto_default_from_file() do any preprocessing of
the lines that it reads. It merely splits the lines at '\n'
and utf8safe-unescapes them.
This was previously duplicated also by NMConfigData's property
setter. We don't need to do it twice.
- sort the lines. This makes the entire handling O(n*ln(n)) instead
of O(n^2). Also, sorting effectively normalizes the content, and
it's desirable to have one true representation of what we write.
In particular when calling nm_utils_strv_sort() with a positive length
argument, then this is not a %NULL terminated strv arrary. That may mean
that it makes sense for the input array to contain %NULL strings.
Use a strcmp() function that accepts %NULL too.
While this is not used at the moment, I think nm_utils_strv_sort()
should accept %NULL strings beause otherwise it's a possibly unexpected
restriction of its API. The function should handle sensible input gracefully.
It is like strcmp(), but has a signature suitable for GCompareDataFunc.
This is necessary with nm_utils_ptrarray_find_binary_search()
to search a sorted strv array.
The fault is here really C, which doesn't allow inline static functions.
So, you need all kinds of slightly different flavors for the same
callbacks (with or without user-data).
Note that glib2 internally just casts strcmp() to GCompareDataFunc ([1]),
relying on the fact how arguments are passed to the function and
ignoring the additional user-data argument. But I find that really
ugly and probably not permissible in general C. Dunno whether POSIX
would guarantee for this to work. I'd rather not do such function
pointer casts.
[1] 0c0cf59858/glib/garray.c (L1792)
Using clock_gettime() directly is a bit inconvenient. We usually
want to combine the fields of struct timespec into one timestamp
(for example, in unit nanoseconds).
Add a helper function to do that.
nm_strndup_a() uses strncpy() because we want the behavior of clearing out
the memory after the first NUL byte. But that can cause a compiler warning:
CC src/settings/plugins/keyfile/libNetworkManager_la-nms-keyfile-utils.lo
In file included from ../../shared/nm-default.h:279,
from ../../src/settings/plugins/keyfile/nms-keyfile-utils.c:20:
In function ‘_nm_strndup_a_step’,
inlined from ‘nms_keyfile_loaded_uuid_is_filename’ at ../../src/settings/plugins/keyfile/nms-keyfile-utils.c:65:9:
../../shared/nm-glib-aux/nm-macros-internal.h:1661:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
1661 | strncpy (s, str, len);
| ^~~~~~~~~~~~~~~~~~~~~
../../src/settings/plugins/keyfile/nms-keyfile-utils.c: In function ‘nms_keyfile_loaded_uuid_is_filename’:
../../src/settings/plugins/keyfile/nms-keyfile-utils.c:48:8: note: length computed here
48 | len = strlen (filename);
| ^~~~~~~~~~~~~~~~~
It's true that the len argument of _nm_strndup_a_step() depends on the
string length of the source string. But in this case it's safe, because
we checked that the destination buffer is exactly the right size too.
By that reasoning we should use memcpy() or strcpy(), but both are
unsuitable. That is because we want nm_strndup_a() to behave like
strndup(), which means we need to handle cases where the len argument
is larger than the string length of the source string. That is, we want
always to return a buffer of size len+1, but we want to copy only the
characters up to the first NUL byte, and clear out the rest. That's what
strncpy() does for us.
Silence the warning.
If there is only one argument, we can assume this is a plain string.
That is especially the case, because g_set_error() is G_GNUC_PRINTF()
and would warn if this would be a format string with missing parameters.
This is for convenience. Previously, one was compelled to explicitly
choose between nm_utils_error_set_literal() and nm_utils_error_set().
Now, it automatically chooses.
Note that there are a few things that won't work, like
nm_utils_error_set (error, code, "bogus %u escape");
But that's good. You get a compiler warning (as you used to)
and it's clear in this case you really need
nm_utils_error_set_literal().
This follows a pointer to a pointer and compares them. In a sense
it's like nm_pstr_*(), which follow a pointer to a string. However,
these functions use direct pointer comparison.
The purpose is when you hash a key that has as first field a pointer
value (and then compare them by pointer equality).
The probe takes a reference to the current lease and so it must
release it upon destruction.
Signed-off-by: Beniamino Galvani <bgalvani@redhat.com>
https://github.com/nettools/n-dhcp4/pull/1
Add nm_sd_dns_name_to_wire_format() based on systemd utilities to
convert a name into its wire format according to RFC 1035 section
3.1. It will be used to build the content of the DHCP FQDN option.
dns-domain.c contains useful functions for manipulating DNS names.
Add it to the systemd static library we build in shared/, similarly to
what we already do for other utility files that were originally in
src/systemd/src/basic/.
In particular, avoid including linux/netdevice.h from headers. This is
not a problem on newer distros, but required for CentOS 7.6.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This is inspired by the existing systemd integration, with a few differences:
* This parses the WPAD option, which systemd requested, but did not use.
* We hook into the DAD handling, only making use of the configured address
once DAD has completed successfully, and declining the lease if it fails.
There are still many areas of possible improvement. In particular, we need
to ensure the parsing of all options are compliant, as n-dhcp4 treats all
options as opaque, unlike sd-dhcp4. We probably also need to look at how
to handle failures and retries (in particular if we decline a lease).
We need to query the current MTU at client startu, as well as the hardware
broadcast address. Both these are provided by the kernel over netlink, so
it should simply be a matter of hooking that up with NM's netlink layer.
Contribution under LGPL2.0+, in addition to stated licenses.
Previously, nm_c_list_move_*() only allowed to move element inside the
same list. Relax that, it works just the same list to move the element
from one list into a different list.
Usually we avoid GSList, because I think it's not a great data type.
Anyway, our match-specs are just a GSList of strings, so we need some
API to handle them.
We no longer add these. If you use Emacs, configure it yourself.
Also, due to our "smart-tab" usage the editor anyway does a subpar
job handling our tabs. However, on the upside every user can choose
whatever tab-width he/she prefers. If "smart-tabs" are used properly
(like we do), every tab-width will work.
No manual changes, just ran commands:
F=($(git grep -l -e '-\*-'))
sed '1 { /\/\* *-\*- *[mM]ode.*\*\/$/d }' -i "${F[@]}"
sed '1,4 { /^\(#\|--\|dnl\) *-\*- [mM]ode/d }' -i "${F[@]}"
Check remaining lines with:
git grep -e '-\*-'
The ultimate purpose of this is to cleanup our files and eventually use
SPDX license identifiers. For that, first get rid of the boilerplate lines.
nmtst_get_rand_int() was originally named that way, because it
calls g_rand_int(). But I think if a function returns an uint32, it
should also be named that way.
Rename.
Clang 3.4.2-9.el7 on CentOS7.6 complains about missing generic type match:
../dispatcher/nm-dispatcher.c:243:2: error: controlling expression type 'const Request *const' (aka 'const struct Request *const') not compatible with any generic association type
_LOG_R_D (request, "start running ordered scripts...");
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Fixes: 17dc6a9da6 ('shared: add _NM_ENSURE_TYPE_CONST()')
The sole purpose of this is more type-safe macros.
An alternative solution would be to define a function instead of a
macro. But if the function is unused (currently!) you get a compiler
warning (on some compilers even when marking the function as "static
inline", if it's in the source file). A workaround for that would be
to mark the function as _nm_unused, or to use a macro instead.
_NM_ENSURE_TYPE_CONST() is to aid the macro solution.
NMTeamLinkWatcher is immutable, meaning there is no way to change an
existing instance (aside increasing/decreasing the ref-count).
Hence, all API is implicitly const.
Still, mark the arguments as const.
