In commit 8a46b25cfa, NetworkManager
bumped its glib dependency to 2.40 or newer.
"nm-glib.h" header is precisely the compatiblity implementation that we
use to cope with older glib versions. Note, that the same implementation
is also used by applet and VPN plugins.
However, applet and VPN plugins don't yet require 2.40 glib. Also,
they don't yet require NetworkManager 1.12.0 API (which was the one
that bumped the glib requirement to 2.40). Hence, when "nm-glib.h"
is used in applet or VPN, it must still cope with older versions,
although, the code is not used by NetworkManager itself.
Partly revert 8a46b25cfa so that nm-glib.h
again works for 2.32 or newer.
The same is true, also for "nm-test-utils.h", which is also used by
applet and VPN plugins.
On m68k architecture, the struct
typedef struct {
gint64 timestamp_ms;
bool dirty;
} IP6RoutesTemporaryNotAvailableData;
ends up being of a previously unhandled size. Causing a compile time
assertion to fail.
Support argument sizes of 10 bytes for nm_g_slice_free_fcn().
Also, assign *_pp before unref-ing the old value. Calling
g_object_unref() on the old value, might invoke callbacks
that are out of control of nm_g_object_ref_set(). During
that time, the pointer should already be assigned the new value,
instead of having an intermediate %NULL value. In most cases,
this would of course not matter, but there is no need to let
anyone see an intermediate %NULL value for a moment.
Also, don't use typeof(**_pp), which would not work with opaque
types (like we commonly have).
The files in shared/nm-utils are not compiled as one static library,
instead each subproject that needs (parts of) them, re-compiles the
files individually.
The major reason for that is, because we might have different compile
flags, depending on whether we build libnm-core or
libnm-util/libnm-glib. Actually, I think that is not really the case,
and maybe this should be refactored, to indeed build them all as a
static library first.
Anyway, libnm-util, libnm-glib, clients' common lib, they all need a
different set of shared files that they should compile. Refactor
"shared/meson.build" to account for that and handle it like autotools
does.
Another change is, that "shared_c_siphash_dep" no longer advertises
"include_directories: include_directories('c-siphash/src')". We don't
put c-siphash.h into the include search path. Users who need it, should
include it via "#include <c-siphash/src/c-siphash.h>". The only exception
is when building shared_n_acd library, which is not under our control.
Originally, we used "nm-utils/siphash24.c", which was copied
from systemd's source tree. It was both used by our own NetworkManager
code, and by our internal systemd fork.
Then, we added "shared/c-siphash" as a dependency for n-acd.
Now, drop systemd's implementation and use c-siphash also
for our internal purpose. Also, let systemd code use c-siphash,
by patching "src/systemd/src/basic/siphash24.h".
Use two common defines NM_BUILD_SRCDIR and NM_BUILD_BUILDDIR
for specifying the location of srcdir and builddir.
Note that this is only relevant for tests, as they expect
a certain layout of the directories, to find files that concern
them.
All users are supposed to include files from nm-utils by fully specifying
the path. -I.*shared/nm-utils is wrong.
Only, systemd code likes to include "siphash24.h" directly. Instead of
adding "-Ishared/nm-utils" to the search path, add an intermediary
header to sd-adapt. Note, that in the meantime we anyway should rework
siphash24 to use shared/c-siphash instead.
This also fixes build for meson, which was broken recently.
This was only used for some extra assertions. It' is not essential.
If this would be for real usage, we should add a dependancy so that
nm-utils/nm-enum-utils.c requires nm-hash-utils.h. But as it is,
this is not necessary.
This fixes build for meson, which wrongly tries to build nm-enum-utils.c
for libnm-util, but then fails to include nm-hash-utils.c. That should
be fixed independently.
Fixes: 84a6eff106
For _nm_utils_enum_to_str_full(), we always first look whether we have
an alias/nick for the numeric value, and preferably use that. That makes a
lot of sense, as it allows the caller to provide better names (aliases),
which are preferred over the name from the GLib type. It renames the
numeric value.
For the reverse conversion, this makes less sense. A name should have a
unique numeric value. That is, we should not use one name that maps to
a different numeric value based on value_infos and GLib type. IOW, we
should not re-number names.
Add an assertion that we don't provide such a value_infos parameter,
that conflicts with names from GLib type.
Also, although the case where GLib type and value_infos disagree is now
forbidden by an assert, reorder the statements in _nm_utils_enum_from_str_full()
too. There is no difference in practice, but it mirros what we do in the
to-str case.
NM sometimes brings an interface temporarily down (for example to
change a VLAN MAC to align it to the parent interface's one). When
this happens, any recv() or send() in n-acd fails, the n-acd instance
is reset to the initial state and a DOWN event is reported to the
manager, which currently does not handle it. The result is an
inconsistent state.
There is no simple way of dealing with the DOWN event in the
manager. What we can do instead is to:
- ignore errors during recv() because there is really nothing we can
do, except for waiting timeouts to expire;
- during probe, ignore errors during send() so that we don't exceed
the probe timeout;
- during announcement, retry after a send() error to ensure we send
all 3 announcements.
https://bugzilla.redhat.com/show_bug.cgi?id=1578675
When doing announcements, use the the timeout specified by RFC
5227. Note that timeout_multiplier might be 0.
This aligns behavior to upstream version of n-acd.
At various places we sort our D-Bus paths. For example,
server sorts them before exporting them on D-Bus.
Server knows well, that a lot of these paths are build
by attaching an incrementing number as last component.
It looks nicer to sort by this number, instead of strictly
lexical with strcmp().
Note that this handles the cases correctly where paths have
different prefixes, or where they don't end with a number.
tools/test-networkmanager-service.py is our NetworkManager stub server.
NetworkManager uses libnm(-core) heavily, for example to decide whether
a connection verifies (nm_connection_verify()) and for normalizing
connections (nm_connection_normalize()).
If the stub server wants to mimic NetworkManager, it also must use these
function. Luckily, we already can do so, by loading libnm using python
GObject introspection.
We already correctly set GI_TYPELIB_PATH search path, so that the
correct libnm is loaded -- provided that we build with introspection
enabled.
We still need to gracefully fail, if starting the stub server fails.
That requries some extra effort. If the stub server notices that
something is missing, it shall exit with status 77. That will cause
the tests to g_test_skip().
Coccinelle:
@@
expression a, b;
@@
-a ? a : b
+a ?: b
Applied with:
spatch --sp-file ternary.cocci --in-place --smpl-spacing --dir .
With some manual adjustments on spots that Cocci didn't catch for
reasons unknown.
Thanks to the marvelous effort of the GNU compiler developer we can now
spare a couple of bits that could be used for more important things,
like this commit message. Standards commitees yet have to catch up.
It is meant to be rather similar in nature to isblank() or
g_ascii_isspace().
Sadly, isblank() is locale dependent while g_ascii_isspace() also considers
vertical whitespace as a space. That's no good for configuration files that
are strucutured into lines, which happens to be a pretty common case.
...so that its prototype is compatible with GDestroyNotify:
src/devices/nm-acd-manager.c: In function ‘destroy_address_info’:
/usr/include/glib-2.0/glib/gmem.h:120:31: error: cast between incompatible function types from ‘NAcd * (*)(NAcd *)’ {aka ‘struct NAcd * (*)(struct NAcd *)’} to ‘void (*)(void *)’ [-Werror=cast-function-type]
GDestroyNotify _destroy = (GDestroyNotify) (destroy); \
^
src/devices/nm-acd-manager.c:430:2: note: in expansion of macro ‘g_clear_pointer’
g_clear_pointer (&info->acd, n_acd_free);
^~~~~~~~~~~~~~~
The same change was done upstream, so the subsequent subtree pull of n-acd
won't mess this up.
For one, these functions are not often needed. No need to define them in the
"nm-macros-internal.h" header, which is included everywhere. Move them to
"nm-shared-utils.h", which must be explicitly included.
Also, these functions are usually not called directly, but by passing their
function pointer to a sort function or similar. There is no point in having
defined in the header file.
If the main loop is quit before the timeout expires, we leave the
timeout source running on the main loop context. Since we usually
create the main loop using the default context, the source will fire
on the next main loop we create during the test.
Therefore, destroy the timeout source if it is still active.
Fixes: 766f31507b
The README states that a kernel >= 3.0 is enough, however
CLOCK_BOOTTIME is only available since kernel 3.15.
Fall back to CLOCK_MONOTONIC when CLOCK_BOOTTIME is not available.
See: https://github.com/nettools/n-acd/pull/3
Signed-off-by: Beniamino Galvani <bgalvani@redhat.com>
Eventually, we should replace our uses of libgsystem's gsystem-local-alloc.h
by glib's g_auto*. As a first tiny step, add a compat implementation for g_autofree,
so that we could at least go ahead and use it instead of gs_free.
https://bugzilla.gnome.org/show_bug.cgi?id=794294
Previously, NM_PTRARRAY_LEN() would not work if the pointer type is
an opaque type, which is common. For example:
NMConnection *const*connections = ...;
Add an alternative to g_clear_pointer(). The differences are:
- nm_clear_pointer() is more type safe as it does not cast neither the
pointer nor the destroy function. Commonly, the types should be compatible
and not requiring a cast. Casting in the macro eliminates some of the
compilers type checking. For example, while
g_clear_pointer (&priv->hash_table, g_ptr_array_unref);
compiles, nm_clear_pointer() would prevent such an invalid use.
- also, clear the destination pointer *before* invoking the destroy
function. Destroy might emit signals (like weak-pointer callbacks
of GArray clear functions). Clear the destination first, so that
we don't leave a dangling pointer there.
- return TRUE/FALSE depending on whether there was a pointer to clear.
I tested that redefining g_clear_pointer()/g_clear_object() with our
more typesafe nm_* variants still compiles and indicates no bugs. So
that is good. It's not really expected that turning on more static checks
would yield a large number of bugs, because generally our code is in a good
shape already. We have few such bugs, because we already turn all all warnings
and extra checks that make sense. That however is not an argument for
not introducing (and using) a more resticted implementation.
It's slightly more correct to first clear the pointer location
before invoking the destroy function. The destroy function might
emit other callbacks, and at a certain point the pointer becomes
dangling. Avoid this danling pointer, by first clearing the
memory, and then destroing the instance.
Add macros that cast away the constness of a pointer, but
ensure that the type of the pointer is as expected.
Unfortunately, there is no way (AFAIK) to remove the constness of
a variable, without explicitly passing @type to the macro.
GCC 8.0's -Wcast-function-type objects casting function pointers to ones
with incompatible prototypes. Sometimes we do that on purpose though.
Notably, the g_source_set_callback()'s func argument can point to functions
of various prototypes. Also, libnm-glib/nm-remote-connection is perhaps
just not worth reworking, that would just be a waste of time.
A cast to void(*)(void) avoids the GCC warning, let's use it.
This makes its prototype compatible with GDestroyNotify so that GCC 8.0
won't warn.
The return value is not used anywhere and the unref() functions typically
don't return any.
When pushing a warning disable with clang, always disable
-Wunknown-warning-option first -- it might be that clang wouldn't warn
of what we're trying to disable because it doesn't recognize it in the
first place. That is entierely okay.
With clang-5.0.0:
CC libnm/tests/libnm_tests_test_secret_agent-test-secret-agent.o
In file included from libnm/tests/test-secret-agent.c:29:
In file included from ./shared/nm-test-libnm-utils.h:23:
./shared/nm-utils/nm-test-utils.h:432:3: error: unknown warning group '-Wunused-but-set-variable', ignored [-Werror,-Wunknown-warning-option]
NM_PRAGMA_WARNING_DISABLE("-Wunused-but-set-variable")
^
./shared/nm-utils/nm-macros-internal.h:223:9: note: expanded from macro 'NM_PRAGMA_WARNING_DISABLE'
_Pragma(_NM_PRAGMA_WARNING_DO(warning))
^
<scratch space>:204:25: note: expanded from here
GCC diagnostic ignored "-Wunused-but-set-variable"
^
1 error generated.
NM_API_VERSION is a better name. It's not the current stable
version, but the version number of the API which the current
NM_VERSION provides. In practice, NM_API_VERSION is either identical
to NM_VERSION (in case of a release) or NM_VERSION is a development
version leading up the the upcoming NM_API_VERSION.
For example, with the new name the check
#if NM_VERSION != NM_API_VERSION
# warning this is an development version
#endif
makes more sense.
We have a well defined versioning scheme and a defined way how
the version number indicates a stable version. We can simply
calculate NM_VERSION_CUR_STABLE based on the version numbers.
It already defaults to the right value. We only need to define
NM_VERSION_MIN_REQUIRED, so that parts of our internal build
can make use of deprecated API.
We don't need to have two version defines "CUR" and "NEXT".
The main purpose of these macros (if not their only), is to
make NM_AVAILABLE_IN_* and NM_DEPRECATED_IN_* macros work.
1) At the precise commit of a release, "CUR" and "NEXT" must be identical,
because whenever the user configures NM_VERSION_MIN_REQUIRED and
NM_VERSION_MAX_ALLOWED, then they both compare against the current
version, at which point "CUR" == "NEXT".
2) Every other commit aside the release, is a development version that leads
up the the next coming release. But as far as versioning is concerned, such
a development version should be treated like that future release. It's unstable
API and it may or may not be close to later API of the release. But
we shall treat it as that version. Hence, also in this case, we want to
set both "NM_VERSION_CUR_STABLE" and again NEXT to the future version.
This makes NM_VERSION_NEXT_STABLE redundant.
Previously, the separation between current and next version would for
example allow that NM_VERSION_CUR_STABLE is the previously release
stable API, and NM_VERSION_NEXT_STABLE is the version of the next upcoming
stable API. So, we could allow "examples" to make use of development
API, but other(?) internal code still restrict to unstable API. But it's
unclear which other code would want to avoid current development.
Also, the points 1) and 2) were badly understood. Note that for our
previousy releases, we usually didn't bump the macros at the stable
release (and if we did, we didn't set them to be the same). While using
two macros might be more powerful, it is hard to grok and easy to
forget to bump the macros a the right time. One macro shall suffice.
All this also means, that *immediately* after making a new release, we shall
bump the version number in `configure.ac` and "NM_VERSION_CUR_STABLE".
Some targets are missing dependencies on some generated sources in
the meson port. These makes the build to fail due to missing source
files on a highly parallelized build.
These dependencies have been resolved by taking advantage of meson's
internal dependencies which can be used to pass source files,
include directories, libraries and compiler flags.
One of such internal dependencies called `core_dep` was already in
use. However, in order to avoid any confusion with another new
internal dependency called `nm_core_dep`, which is used to include
directories and source files from the `libnm-core` directory, the
`core_dep` dependency has been renamed to `nm_dep`.
These changes have allowed minimizing the build details which are
inherited by using those dependencies. The parallelized build has
also been improved.
A cmp() implementation, for sorting an array with pointers, where each
pointer is an inteter according to GPOINTER_TO_INT().
That cames for example handy, if you have a GHashTable with keys
GINT_TO_POINTER(). Then you get the list of keys via
g_hash_table_get_keys_as_array() and want to sort them.
Sometimes, we want to use CList to track a simple data item. But contrary
to GList/GSList, we need to define a structure to hold the data pointer
and the CList member.
Add a generic NMCListElem type that can be used for such simple uses.
Before you ask: why not use GList/GSList? Because even simple operations
like g_list_append() is O(n), which kinda defeats the purpose of having
a doubly linked list.
This code is added to a new header file nm-c-list.h, the reason is that
there is no other good place:
- "nm-utils/c-list.h" is a clone of upstream, it should not deviate.
- "nm-utils/c-list-util.h" contains our utils functions for c-list.h
but should be plain C, independent of glib.
- "nm-utils/nm-shared-utils.h" contains our glib related utilities,
but it should not drag in "c-list.h".
So, "nm-c-list.h" is a utility libray that extends "c-list.h" and
requires glib.
Note that:
- we compile some source files multiple times. Most notably those
under "shared/".
- we include a default header "shared/nm-default.h" in every source
file. This header is supposed to setup a common environment by defining
and including parts that are commonly used. As we always include the
same header, the header must behave differently depending
one whether the compilation is for libnm-core, NetworkManager or
libnm-glib. E.g. it must include <glib/gi18n.h> or <glib/gi18n-lib.h>
depending on whether we compile a library or an application.
For that, the source files need the NETWORKMANAGER_COMPILATION #define
to behave accordingly.
Extend the define to be composed of flags. These flags are all named
NM_NETWORKMANAGER_COMPILATION_WITH_*, they indicate which part of the
build are available. E.g. when building libnm-core.la itself, then
WITH_LIBNM_CORE, WITH_LIBNM_CORE_INTERNAL, and WITH_LIBNM_CORE_PRIVATE
are available. When building NetworkManager, WITH_LIBNM_CORE_PRIVATE
is not available but the internal parts are still accessible. When
building nmcli, only WITH_LIBNM_CORE (the public part) is available.
This granularily controls the build.
This is still the very same approach (in the way the array is split
and how elements are compared). The only difference is that the
recursive implementation is replaced by a non-recursive one.
It's (still) stable, top-down merge-sort.
The non-recursive implementation better, because it avoids the overhead
of the function call to recurse.
The compat implementations return a (transfer none) strv instead of a
(transfer container) one. This has caused double frees in nm-applet:
https://bugs.archlinux.org/task/56772
Don't copy the keys and don't free the container later.
[thaller@redhat.com: patch adjusted to avoid compiler warning]
Fixes: 272439cb20
At several places we create strv arrays where the
strings themself are not deep-copied.
This helper function iterates over such an "const char **"
array, clones the strings, and updates the strv array
inplace to be a "char **" strv array.
This helper function is to reduce code duplication.
At various places we get the (string) keys of a GHashTable.
Add a helper function that does that, including an argument
for optional sorting.
The helper function is there to get reduce code duplication.
Systemd instroduces a macro _fallthrough_, see
https://github.com/systemd/systemd/pull/7389.
However, it does not yet seem conclusive how to
handle this properly in ever situation.
While shared/nm-utils/siphash24.c makes use of
the new macro, don't do that in our fork. siphash24.h
does not include all systemd headers, hence _fallthrough_
is not defined. We could re-implement it as _nm_fallthrough,
but given the open questions, that doesn't seem the
Systemd introduced a _fallthrough_ macro in
https://github.com/systemd/systemd/pull/7389.
There might still be some issue with it, but as
I am going to re-import the latest systemd code,
we get them too.
We need it, because "shared/nm-utils/siphash24.c"
will use it too, and that source file does not include
the other systemd macros. So, we will need to re-define
it.
NMUtilsNamedValue's purpose is precisely to create
a list and sort by entires.
Add nm_utils_named_values_from_str_dict() as helper
function to do that.
It is common to have some data indexed by a name.
If you want to sort a list of such data, you would
have to re-implement your own compare function each time.
Instead, add NMUtilsNamedEntry which as first field has
the name. So, you can create your own struct:
struct my_data {
const char *name;
... other fields
}
and compare them with with nm_utils_named_entry_cmp().
For convenience, add another struct NMUtilsNamedValue, which
has only one data field, a pointer.
This allows the compiler to inline the siphash24*() functions
for nm_hash_ptr() and nm_hash_str() (even without LTO).
This of course only applies to nm_hash_ptr() and nm_hash_str(),
which are implemented in "nm-hash-utils.c" itself. All other
nm_hash_*() functions are inline functions in "nm-hash-utils.h",
and thus these functions can be inlined instead. That is, in
other cases, the nm_hash_*() function instead can be inlined.
For nm_hash_ptr() and nm_hash_str() instead we want to inline the
siphash24*() functions.
So, no longer compile "siphash24.c" directly. Instead, only
build "nm-hash-utils.c" which internally #include "siphash24.c".
Next we will use siphash24() instead of the glib version g_direct_hash() or
g_str_hash(). Hence, the "nm-utils/nm-hash-utils.h" header becomes very
fundamental and will be needed basically everywhere.
Instead of requiring the users to include them, let it be included via
"nm-default.h" header.
siphash24() mixes the bits much better then our naive xor.
Don't bypass siphash24(). We supposedly use it for the
better hashing properties, so use it also for pointers.
When using siphash24(), the hash value depends on the hashed input
and the key from _get_hash_key(). If the input is static, so is also
the result of siphash24(), albeit the bits are scrabbled more.
Add a nm_hash_static() to get such a static key, but without actually
doing siphash24(). The static key is also xored with a static_seed.
For that, also mangle the first byte of the hash key using siphash24()
itself. That is, because nm_hash_static() only uses the first guint of the
random key. Hence, we want that this first guint has all the entropy
of the entire key. We use siphash24() itself, to mangle all bits
of the 16 byte key into the first guint.
The nm_close() wrapper should behave exactly the same as calling
close() directly. This is well known, documented behavior.
The only addition on top of that, should be the nm_assert() to catch
double-closing.
Prevously, when passing a negative file descriptor, we wouldn't properly
set errno. Also, the call would not show up in strace, which it should
(at least, if libc's close actually makes the syscall).
I would argue, that passing a negative file descriptor is a bug already
and we should never do that. Maybe we should even assert non-negative
fds. I don't do that now, because I am not sufficiently confident.
Anyway, the change should have not practical effect, because we
shouldn't actually pass negative fds already.
There is still a fallback detection in "shared/nm-utils/nm-macros-internal.h",
so that VPN-plugins and applet don't need to bother about adding these
configure checks.
The _NM_GET_PRIVATE() macro already preserved and propagated
the constness of @self to the resulting private pointer.
_NM_GET_PRIVATE_PTR() didn't do that. Extend the macro,
to make that possible.
- nm-ovsdb.c uses json_load_callback(), which is jansson v2.4.
Hence, it cannot build the OVS plugin in our Travis-CI, which is
still on Ubuntu Precise. Disable building the plugin in travis and
add a compiler warning when building against an older version.
- since jansson v2.3, there is json_object_key_to_iter() to implement
the for-each macros. Use it in json_object_foreach_safe() when
available.
We need to pass more alias-types. Instead of having numbered
versions, use variadic number of macro arguments.
Also, fix build failure with old compiler:
In file included from src/nm-ip6-config.c:24:
./src/nm-ip6-config.h:44:29: error: controlling expression type 'typeof (ipconf_iter->current->obj)' (aka 'const void *const') not compatible with any generic association type
*out_address = has_next ? NMP_OBJECT_CAST_IP6_ADDRESS (ipconf_iter->current->obj) : NULL;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Fixes: b1810d7a68
_NM_GET_PRIVATE() used typeof() to propagate constness of the @self
pointer. However, that means, it could only be used with a self pointer
of the exact type. That means, you explicitly had to cast from (GObject *)
or from (void *).
The requirement is cumbersome, and often led us to either create @self
pointer we didn't need:
NMDeviceVlan *self = NM_DEVICE_VLAN (device);
NMDeviceVlanPrivate *priv = NM_DEVICE_VLAN_GET_PRIVATE (self);
or casting:
NMDeviceVlanPrivate *priv = NM_DEVICE_VLAN_GET_PRIVATE ((NMDevice *) device);
In both cases we forcefully cast the source variable, loosing help from
the compiler to detect a bug.
For "nm-linux-platform.c", instead we commonly have a pointer of type
NMPlatform. Hence, we always forcefully cast the type via _NM_GET_PRIVATE_VOID().
Rework the macro to use _Generic(). If compiler supports _Generic(), then we
will get all compile time checks as desired. If the compiler doesn't support
_Generic(), it will still work. You don't get the compile-time checking of course,
but you'd notice that something is wrong once you build with a suitable
compiler.
shared/nm-utils/nm-hash-utils.c:110:3: error: right shift count >= width of type [-Werror]
h = h ^ ((guint) (((uintptr_t) ptr) >> 32)) ^ ((guint) ((uintptr_t) ptr));
^
Even if the branch is not reached on 32-bit architectures, the
compiler still emits a warning for the 32-bit right shift.
Fixes: ee76b0979f
nm_close() is like close(), but throws an assertion if the input fd is
>=0 and invalid. Passing an invalid (i.e. already closed) fd to
close() is a programming error with potentially catastrophic effects,
as another thread may reuse the closed fd.
We often want to cascade hashing, meaning, to combine the
outcome of various hash functions in a larger hash.
Instead of having each hash function return a guint hash value,
accept a hash state argument. This saves the overhead of initializing
and completing the intermediate hash states.
It also avoids loosing entropy when we reduce the larger hash state
into the intermediate guint hash value.
By using a macro, we don't cast all the types to guint. Instead,
we use their native types directly. Hence, we don't need
nm_hash_update_uint64() nor nm_hash_update_ptr().
Also, for types smaller then guint like char, we save hashing
the all zero bytes.
siphash24() is wildly used by projects nowadays.
It's certainly slower then our djb hashing that we used before.
But quite likely it's fast enough for us, given how wildly it is
used. I think it would be hard to profile NetworkManager to show
that the performance of hash tables is the issue, be it with
djb or siphash24.
Certainly with siphash24() it's much harder to exploit the hashing
algorithm to cause worst case hash operations (provided that the
seed is kept private). Does this better resistance against a denial
of service matter for us? Probably not, but let's better be safe then
sorry.
Note that systemd's implementation uses a different seed for each hash
table (at least, after the hash table grows to a certain size).
We don't do that and use only one global seed.
The privious NM_HASH_* macros directly operated on a guint value
and were thus close to the actual implementation.
Replace them by adding a NMHashState struct and accessors to
update the hash state. This hides the implementation better
and would allow us to carry more state. For example, we could
switch to siphash24() transparently.
For now, we still do a form basically djb2 hashing, albeit with
differing start seed.
Also add nm_hash_str() and nm_str_hash():
- nm_hash_str() is our own string hashing implementation
- nm_str_hash() is our own string implementation, but with a
GHashFunc signature, suitable to pass it to g_hash_table_new().
Also, it has this name in order to remind you of g_str_hash(),
which it is replacing.
"nm-utils/nm-shared-utils.h" shall contain utility function without other
dependencies. It is intended to be used by other projects as-is.
nm_utils_random_bytes() requires getrandom() and a HAVE_GETRANDOM configure
check. That makes it more cumbersome to re-use "nm-shared-utils.h", in
cases where you don't care about nm_utils_random_bytes().
Split nm_utils_random_bytes() out to a separate file.
Same for hash utils, which depend on nm_utils_random_bytes(). Also, hash
utils will eventually be extended to use siphash24.
This makes hashing non-deterministic with the aim to
make it harder to exploit hash collisions.
Non-deterministic also means that for unit testing
we will get different values on each run. But since we
shall never assign any meaning to these hash values
nor rely on them being stable between restarts (or
upgrades), that doesn't hurt.
Introduce a NM_HASH_INIT() function. It makes the places
where we initialize a hash with a certain seed visually clear.
Also, move them from "shared/nm-utils/nm-shared-utils.h" to
"shared/nm-utils/nm-macros-internal.h". We might want to
have NM_HASH_INIT() non-inline (hence, define it in the
source file).
Add a new function nm_utils_random_bytes().
This function now preferably uses getrandom() syscall if it is
available.
As fallback, it always tries to fill the buffer from /dev/urandom.
If it cannot, as last fallback it uses GRand, which cannot fail.
Hence, the function always sets some (pseudo) random bytes.
It also returns FALSE if the obtained bytes are possibly not good
randomness.
Fixes the following:
shared/nm-utils/nm-shared-utils.c:136: Warning: NetworkManager: GTK-Doc comment block end token "*/" should not be preceded by comment text:
* Returns: the input buffer with the quoted string. */
We already have nm_strquote_a(). That is useful, but uses alloca(), hence it
is ill suited to be called from a macro, inside a loop, or from a function
that should be inlined.
Instead, add nm_strquote() that has the same purpose but writes to a provided
string buffer.
nm_dedup_multi_obj_ref() is a trivial function, that only uses the field
which is already declared in the same header file. Move it to the header
so that it can be inlined (without LTO).
A replacement for g_strsplit_set(). While g_strsplit_set()
does (n+1) malloc and n slice allocations, this needs
roughtly (O(log(n))) mallocs.
Another difference from g_strsplit_set() is that this function
treats multiple delimiters as one (and thus never returns empty
words). While I can see that sometimes you may want to keep empty
words (like parsing a CSV file and preserve empty cells), we usually
use this function for splitting user input. In such case, we want
to treat multiple delimiters as one.
Kernel does not allow to add IPv6 routes with "src", as long as the
corresponding address is still tentative (related bug rh#1457196).
The workaround for this is cumbersome. First, when we fail to add such a
route with "pref_src", we guess that it happend due to this issue. In
that case, nm_ip6_config_commit() returns the list of routes that could
not be added for the moment (but hopefully can be added later).
We track this list in NMDevice, and keep trying to merge the routes
back into ip6_config. In order to not try indefinitely, keep track of a
timestamp when we tried to add this route for the first time.
Another uglyness is that pending tentative routes don't explicitly block
activation. In practice they may do, because for these routes we also have
an IPv6 address that is still doing DAD, so the IP configuration is
still pending due to that.
https://bugzilla.redhat.com/show_bug.cgi?id=1452684
- nm_clear_g_object() is like g_clear_object() but:
- it returns a boolean value, indicating that something was cleared.
- it includes an nm_assert() to check that the pointer is still
valid.
- it uses typeof() instead of blindly casting the argument.
- nm_g_object_ref_set() combines nm_clear_g_object() and resetting
the pointer to a new object, including taking a reference.
- also returns a boolean, indicating whether something changed.
- it gets the order of operations right: first it increses the
ref-count, before unrefing the old object.
- like nm_clear_g_object() and nm_clear_g_free() it first sets
the destination to NULL, instead of leaving a dangling pointer
for the duraction of the unref/free call.
- fix nm_clear_g_free() not to use a possibly dangling pointer.
Striclty speaking, that is undefined behavior.
And relax the type for nm_auto_unref_gtypeclass macro. Like
g_type_class_unref() itself, you usually don't use it with a GTypeClass
base class, but some subtype like GObjectClass.
For RFC1918 private IPv4addresses, guess a better prefix length for
addresses and routes.
nmtui is an interactive program. It makes sense to be a bit smarter
about what the user probably meant.
It would be nice if nmtui would update the entry field immediately when
the cursor leaves the field, to show the guessed prefix length. However,
that is not easily possible, so lets to that another time.
For IPv6 addresses, default to /64 instead of /128.
https://bugzilla.redhat.com/show_bug.cgi?id=1474295
We rely on clearing the dirty flag. For example in nm_ip4_config_replace(),
we first mark all entries dirty, then force-append the ones we keep,
and finally remove the ones that are still dirty.
Since _nm_ip_config_add_obj() short-cuts nm_dedup_multi_index_add_full(),
it must clear the dirty flag on its own.
Compiler wouldn't recognize that the @route/@address argument is always
initialized. The right workaround seems to let the next() functions always
set the value.
In file included from src/nm-ip6-config.c:24:0:
src/nm-ip6-config.c: In function ‘nm_ip6_config_create_setting’:
src/nm-ip6-config.c:734:62: error: the address of ‘address’ will always evaluate as ‘true’ [-Werror=address]
nm_ip_config_iter_ip6_address_for_each (&ipconf_iter, self, &address) {
^
src/nm-ip6-config.h:60:17: note: in definition of macro ‘nm_ip_config_iter_ip6_address_for_each’
for (({ if (address) { *(((const NMPlatformIP6Address **) address)) = NULL; } }), nm_ip_config_iter_ip6_address_init ((iter), (self)); \
^
Fixes: 6e9aa9402a
Previously, we would add exclusive routes via netlink message flags
NLM_F_CREATE | NLM_F_REPLACE for RTM_NEWROUTE. Similar to `ip route replace`.
Using that form of RTM_NEWROUTE message, we could only add a certain
route with a certain network/plen,metric triple once. That was already
hugely inconvenient, because
- when configuring routes, multiple (managed) interfaces may get
conflicting routes (multihoming). Only one of the routes can be actually
configured using `ip route replace`, so we need to track routes that are
currently shadowed.
- when configuring routes, we might replace externally configured
routes on unmanaged interfaces. We should not interfere with such
routes.
That was worked around by having NMRouteManager (and NMDefaultRouteManager).
NMRouteManager would keep a list of the routes which NetworkManager would like
to configure, even if momentarily being unable to do so due to conflicting routes.
This worked mostly well but was complicated. It involved bumping metrics to
avoid conflicts for device routes, as we might require them for gateway routes.
Drop that now. Instead, use the corresponding of `ip route append` to configure
routes. This allows NetworkManager to confiure (almost) all routes that we care.
Especially, it can configure all routes on a managed interface, without
replacing/interfering with routes on other interfaces. Hence, NMRouteManager
becomes obsolete.
It practice it is a bit more complicated because:
- when adding an IPv4 address, kernel will automatically create a device route
for the subnet. We should avoid that by using the IFA_F_NOPREFIXROUTE flag for
IPv4 addresses (still to-do). But as kernel may not support that flag for IPv4
addresses yet (and we don't require such a kernel yet), we still need functionality
similar to nm_route_manager_ip4_route_register_device_route_purge_list().
This functionality is now handled via nm_platform_ip4_dev_route_blacklist_set().
- trying to configure an IPv6 route with a source address will be rejected
by kernel as long as the address is tentative (see related bug rh#1457196).
Preferably, NMDevice would keep the list of routes which should be configured,
while kernel would have the list of what actually is configured. There is a
feed-back loop where both affect each other (for example, when externally deleting
a route, NMDevice must forget about it too). Previously, NMRouteManager would have
the task of remembering all routes which we currently want to configure, but cannot
due to conflicting routes.
We get rid of that, because now we configure non-exclusive routes. We however still
will need to remember IPv6 routes with a source address, that currently cannot be
configured yet. Hence, we will need to keep track of routes that
currently cannot be configured, but later may be.
That is still not done yet, as NMRouteManager didn't handle this
correctly either.
For completeness of the API. remove_obj() is basically a shortcut
of nm_dedup_multi_index_lookup_obj() combined with
nm_dedup_multi_index_remove_entry(). As such, it is useful to return
the actually deleted object. Note that the lookup needle @obj is not
necessarily the same instance as the one that will be removed, it's
only an instance that compares equal according to the index's equality
operator.
The return value shall indicate whether the add-call changed anything.
Reordering shall count as a change too.
On the other hand, clearing the dirty flag of the entry does not count
as a change.
Until now, NetworkManager's platform cache for routes used the quadruple
network/plen,metric,ifindex for equaliy. That is not kernel's
understanding of how routes behave. For example, with `ip route append`
you can add two IPv4 routes that only differ by their gateway. To
the previous form of platform cache, these two routes would wrongly
look identical, as the cache could not contain both routes. This also
easily leads to cache-inconsistencies.
Now that we have NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID, fix the route's
compare operator to match kernel's.
Well, not entirely. Kernel understands more properties for routes then
NetworkManager. Some of these properties may also be part of the ID according
to kernel. To NetworkManager such routes would still look identical as
they only differ in a property that is not understood. This can still
cause cache-inconsistencies. The only fix here is to add support for
all these properties in NetworkManager as well. However, it's less serious,
because with this commit we support several of the more important properties.
See also the related bug rh#1337855 for kernel.
Another difficulty is that `ip route replace` and `ip route change`
changes an existing route. The replaced route has the same
NM_PLATFORM_IP_ROUTE_CMP_TYPE_WEAK_ID, but differ in the actual
NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID:
# ip -d -4 route show dev v
# ip monitor route &
# ip route add 192.168.5.0/24 dev v
192.168.5.0/24 dev v scope link
# ip route change 192.168.5.0/24 dev v scope 10
192.168.5.0/24 dev v scope 10
# ip -d -4 route show dev v
unicast 192.168.5.0/24 proto boot scope 10
Note that we only got one RTM_NEWROUTE message, although from NMPCache's
point of view, a new route (with a particular ID) was added and another
route (with a different ID) was deleted. The cumbersome workaround is,
to keep an ordered list of the routes, and figure out which route was
replaced in response to an RTM_NEWROUTE. In absence of bugs, this should
work fine. However, as we only rely on events, we might wrongly
introduce a cache-inconsistancy as well. See the related bug rh#1337860.
Also drop nm_platform_ip4_route_get() and the like. The ID of routes
is complex, so it makes little sense to look up a route directly.
Default g_log() logs to stdout for INFO level and higher, but logs to stderr
for DEBUG/TRACE. That is annoying, because especially when redirecting the streams,
the messages get mixed up. Install a log handler and just print to stdout for
the tests.
Reasons:
- it adds an O(1) lookup index for accessing NMIPxConfig's addresses.
Hence, operations like merge/intersect have now runtime O(n) instead
of O(n^2).
Arguably, we expect low numbers of addresses in general. For low
numbers, the O(n^2) doesn't matter and quite likely in those cases
the previous implementation was just fine -- maybe even faster.
But the simple case works fine either way. It's important to scale
well in the exceptional case.
- the tracked objects can be shared between the various NMPI4Config,
NMIP6Config instances with NMPlatform and everybody else.
- the NMPObject can be treated generically, meaning it enables code to
handle both IPv4 and IPv6, or addresses and routes. See for example
_nm_ip_config_add_obj().
- I want core to evolve to somewhere where we don't keep copies of
NMPlatformIP4Address, et al. instances. Instead they shall all be
shared. I hope this will reduce memory consumption (although tracking a
reference consumes some memory too). Also, it shortcuts nmp_object_equal()
when comparing the same object. Calling nmp_object_equal() on the
identical objects would be a common case after the hash function
pre-evaluates equality.
Add a stable, recursive merge sort for CList.
This could be improved by doing an iterative implementation.
The recursive implementation's stack depth is not an issue,
as it is bound by O(ln(n)). But an iterative implementation
would safe the overhead of O(n*log(n)) function calls and be
potentially faster.
And get rid of the unused obj_full_equality_allows_different_class.
It's hard to grasp how to implement different object types that can compare
despite having different klasses. The idea was, that stack allocated
objects (used as lookup needles), are some small lightweight objects,
that still compare equal to the full instance. But it's unused. Drop it.
by moving the core functionality to "nm-dedup-multi.c".
As the ref-counting mechanism now is part of "nm-dedup-multi.c",
this works better and is reusable outside of platform.
Implement the reference counting of NMPObject as part of
NMDedupMultiObj and get rid of NMDedupMultiBox.
With this change, the NMPObject is aware in which NMDedupMultiIndex
instance it is tracked.
- this saves an additional GSlice allocation for the NMDedupMultiBox.
- it is immediately known, whether an NMPObject is tracked by a
certain NMDedupMultiIndex or not. This saves an additional hash
lookup.
- previously, when all idx-types cease to reference an NMDedupMultiObj
instance, it was removed. Now, a tracked objects stays in the
NMDedupMultiIndex until it's last reference is deleted. This possibly
extends the lifetime of the object and we may reuse it better.
- it is no longer possible to add one object to more then one
NMDedupMultiIndex instance. As we anyway want to have only one
instance to deduplicate the objects, this is fine.
- the ref-counting implementation is now part of NMDedupMultiObj.
Previously, NMDedupMultiIndex could also track objects that were
not ref-counted. Hoever, the object anyway *must* implement the
NMDedupMultiObj API, so this flexibility is unneeded and was not
used.
- a downside is, that NMPObject grows by one pointer size, even if
it isn't tracked in the NMDedupMultiIndex. But we really want to
put all objects into the index for sharing and deduplication. So
this downside should be acceptable. Still, code like
nmp_object_stackinit*() needs to handle a larger object.
Add the NMDedupMultiIndex cache. It basically tracks
objects as doubly linked list. With the addition that
each object and the list head is indexed by a hash table.
Also, it supports tracking multiple distinct lists,
all indexed by the idx-type instance.
It also deduplicates the tracked objects and shares them.
- the objects that can be put into the cache must be immutable
and ref-counted. That is, the cache will deduplicate them
and share the reference. Also, as these objects are immutable
and ref-counted, it is safe that users outside the cache
own them too (as long as they keep them immutable and manage
their reference properly).
The deduplication uses obj_id_hash_func() and obj_id_equal_func().
These functions must cover *every* aspect of the objects when
comparing equality. For example nm_platform_ip4_route_cmp()
would be a function that qualifies as obj_id_equal_func().
The cache creates references to the objects as needed and
gives them back. This happens via obj_get_ref() and
obj_put_ref(). Note that obj_get_ref() is free to create
a new object, for example to convert a stack-allocated object
to a (ref-counted) heap allocated one.
The deduplication process creates NMDedupIndexBox instances
which are the ref-counted entity. In principle, the objects
themself don't need to be ref-counted as that is handled by
the boxing instance.
- The cache doesn't only do deduplication. It is a multi-index,
meaning, callers add objects using a index handle NMDedupMultiIdxType.
The NMDedupMultiIdxType instance is the access handle to lookup
the list and objects inside the cache. Note that the idx-type
instance may partition the objects in distinct lists.
For all operations there are cross-references and hash table lookups.
Hence, every operation of this data structure is O(1) and the memory
overhead for an index tracking an object is constant.
The cache preserves ordering (due to linked list) and exposes the list
as public API. This allows users to iterate the list without any
additional copying of elements.
Platform has it's own, simple implementation of object types:
NMPObject. Extract a base type and move it to "shared/nm-utils/nm-obj.h"
so it can be reused.
The base type is trival, but it allows us to implement other objects
which are compatible with NMPObjects. Currently there is no API for generic
NMObjBaseInst type, so compatible in this case only means, that they
can be used in the same context (see example below).
The only thing that you can do with a NMObjBaseInst is check it's
NMObjBaseClass.
Incidentally, NMObjBaseInst is also made compatible to GTypeInstance.
It means, an NMObjBaseInst is not necessarily a valid GTypeInstance (like NMPObject
is not), but it could be implemented as such.
For example, you could do:
if (NMP_CLASS_IS_VALID ((NMPClass *) obj->klass)) {
/* is an NMPObject */
} else if (G_TYPE_CHECK_INSTANCE_TYPE (obj, NM_TYPE_SOMETHING)) {
/* it a NMSometing GType */
} else {
/* something else? */
}
The reason why NMPObject is not implemented as proper GTypeInstance is
because it would require us to register a GType (like
g_type_register_fundamental). However, then the NMPClass struct can
no longer be const and immutable memory. But we could.
NMObjBaseInst may or may not be a GTypeInstance. In a sense, it's
a base type of GTypeInstance and all our objects should be based
on it (optionally, they we may make them valid GTypes too).
Returning TRUE for zero makes no sense. Obviously, zero is not a power
of two.
Also, the function is used to check whether a number has only one bit
(flag) set, so, an alternative name would be "has-one-bit-set", which
also should return FALSE for zero. All callers didn't really care for
the previous meaning "has-at-most-one-bit-set".
This also avoids the issue of checking (x >= 0), which causes
-Wtype-limits warnings for unsigned types. Which was avoided
by doing (x == 0 || x > 0), which caused -Wlogical-op warning,
which then was avoided (x == 0 || (x > 0 && 1)). Just don't.
We recently added -Wlogical-op in our build process
(commit #41e7fca59762dc928c9d67b555b1409c3477b2b0).
Seems that old versions of gcc (4.8.x) will hit that warning with our
implementation of our "nm_utils_is_power_of_two" and
"test_nm_utils_is_power_of_two_do" macros.
Fool it just adding an always TRUE check.
Use C-style backslash escaping to sanitize non-UTF-8 strings.
The functions are compatible with glib's g_strcompress() and
g_strescape().
The difference is only that g_strescape() escapes all non-printable,
non ASCII character as well, while nm_utils_str_utf8safe_escape()
-- depending on the flags -- preserves valid UTF-8 sequence except
backslash.
The flags allow to optionally escape ASCII control characters and
all non-ASCII (valid UTF-8) characters. But the option to preserve
valid UTF-8 (non-ASCII) characters verbatim, is what distinguishes
from g_strescape().
UDev never creates such invalid escape sequences. Anyway,
we cannot accept a NUL character at this point. Just take
the ill escape verbatim -- it should never happen anyway.
Include the circular, doubly-linked list implementation from
c-util/c-list [1], commit 864051de6e7e1c93c782064fbe3a86b4c17ac466.
[1] https://github.com/c-util/c-list
CC shared/nm-utils/libnm_core_libnm_core_la-nm-udev-utils.lo
In file included from ./shared/nm-utils/nm-glib.h:27:0,
from ./shared/nm-utils/nm-macros-internal.h:29,
from ./shared/nm-default.h:178,
from shared/nm-utils/nm-udev-utils.c:21:
shared/nm-utils/nm-udev-utils.c: In function ‘nm_udev_client_enumerate_new’:
./shared/nm-utils/gsystem-local-alloc.h:53:50: error: ‘to_free’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
GS_DEFINE_CLEANUP_FUNCTION(void*, gs_local_free, g_free)
^~~~~~
shared/nm-utils/nm-udev-utils.c:147:18: note: ‘to_free’ was declared here
gs_free char *to_free;
^~~~~~~
In file included from ./shared/nm-utils/nm-glib.h:27:0,
from ./shared/nm-utils/nm-macros-internal.h:29,
from ./shared/nm-default.h:178,
from shared/nm-utils/nm-udev-utils.c:21:
shared/nm-utils/nm-udev-utils.c: In function ‘nm_udev_client_new’:
./shared/nm-utils/gsystem-local-alloc.h:53:50: error: ‘to_free’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
GS_DEFINE_CLEANUP_FUNCTION(void*, gs_local_free, g_free)
^~~~~~
shared/nm-utils/nm-udev-utils.c:243:20: note: ‘to_free’ was declared here
gs_free char *to_free;
^~~~~~~
Fixes: e32839838e
Before refactoring nmcli recently, field names were marked for translation.
Note that for the property names, marking them had no effect as only
plain strings can be marked with N_().
Note how --fields are also an input argument. The input should be
independent of the locale and not translated. Likewise, when printing
the header names, they should not be translated to match the --fields
option.
$ LANG=de_DE.utf8 nmcli --fields GENERAL.DEVICE device show enp0s25
GENERAL.GERÄT: enp0s25
Drop the translation marks.
I used to use g_strv_length ((char **) p) instead, but that feels
ugly because it g_strv_length() is not designed to operate on
arbitrary pointer arrays.
Commit a8730c51c8 moved the enum
utils from libnm-core to shared/nm-utils.
However, three of those functions are part of public API in libnm.
So, when statically linking against "shared/nm-utils/nm-enum-utils.c"
and dynamically linking against libnm.so, those symbols are present
twice and cause a linker failure.
Fix that by moving the public API back to libnm-core.
Fixes: a8730c51c8
libnm contains the public function nm_utils_enum_from_str() et al.
The function is not flexible enough for nmcli's usecase. So, I would
need another public function like nm_utils_enum_from_str_full() that
has an extended API.
That was already required previously for ifcfg-rh writer, but in that
case I could just add it as internal API as libnm-core is linked statically
with NetworkManager.
I don't want to commit to a public API for an utility function. So move
the code instead to the shared directory, so that nmcli may link
statically against it and use the internal API.
These functions are only used by nm-meta-setting-desc.c. Make them internal.
Unfortunately, they are part of "common.h" which cannot be used without
the rest of nmcli. Still todo.
This part contains static functions and variables to describe
settings. It is distinct from the mechanism to use them, or
access them.
Split it out.
It still uses clients/cli/common.h and clients/cli/utils.h
which shall be fixed next.
GUdevClient always creates a monitor instance, even if there are no subsystems
or handlers defined. Hence the first iteration of NMUdevClient did that as
well.
I think that can be avoided however. We only need a monitor when there is
a event handler subscribed. Contrary to GUdevClient, we know that from the
very beginning.
_NM_GET_PRIVATE() macro is used to implement a standard private-getter, but it
requires that "self" is a pointer of either "const type *" or "type *". That
is great in most cases, but sometimes we have predominatly self pointers of
different type, so it would require a lot of casts.
Add a different form _NM_GET_PRIVATE_VOID() where self pointer can be any
non-const pointer and returns a non-const private pointer after casting.
"shared/nm-setting-metadata.h" will contain data structures
to handle NM setting properties in a generic way.
For now, this is internal API, but shared between libnm-core (which
extends to libnm, NetworkManager, device-plugins, settings-plugins),
and nmcli.
Related: https://bugzilla.gnome.org/show_bug.cgi?id=732292
I think NM_CACHED_QUARK_FCN() is better because:
- the implementation is in our hand, meaning it is clear that
putting a "static" before NM_CACHED_QUARK_FCN() is guaranteed to
work -- without relying on G_DEFINE_QUARK() to be defined in a way
that this works (in fact, we currently never do that and instead
make all functions non-static).
- it does not construct function names by appending "_quark".
Thus you can grep for the entire function name and finding
the place where it is implemented.
- same with the stings, where the new macro doesn't stringify the
argument, which is less surpising. Again, now you can grep
for the string including the double quoting.
(yes, I really use grep to understand the source-code)
NM_CACHED_QUARK_FCN() is a replacement for G_DEFINE_QUARK().
G_DEFINE_QUARK() is mostly used to define GError quarks. As
such, it always appends _quark() to the function name, which
is unfavorable because it makes it harder to grep for the
definition of the function.
In general I think that macros that defined symbols by concatenating
something should be avoided because that makes it harder to locate
where the symbol was defined.
