system/NetworkManager
system/NetworkManager
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/NetworkManager/NetworkManager synced 2024-10-14 20:18:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
NetworkManager/shared/nm-glib-aux/nm-dedup-multi.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

1066 lines
36 KiB
C
Raw Normal View History

all: unify comment style for SPDX-License-Identifier tag Our coding style recommends C style comments (/* */) instead of C++ (//). Also, systemd (which we partly fork) uses C style comments for the SPDX-License-Identifier. Unify the style. $ sed -i '1 s#// SPDX-License-Identifier: \([^ ]\+\)$#/* SPDX-License-Identifier: \1 */#' -- $(git ls-files -- '*.[hc]' '*.[hc]pp')
2020-09-29 14:42:22 +00:00
/* SPDX-License-Identifier: LGPL-2.1+ */
all: manually drop code comments with file description
2019-09-25 11:13:40 +00:00
/*
all: unify format of our Copyright source code comments ```bash readarray -d '' FILES < <( git ls-files -z \ ':(exclude)po' \ ':(exclude)shared/c-rbtree' \ ':(exclude)shared/c-list' \ ':(exclude)shared/c-siphash' \ ':(exclude)shared/c-stdaux' \ ':(exclude)shared/n-acd' \ ':(exclude)shared/n-dhcp4' \ ':(exclude)src/systemd/src' \ ':(exclude)shared/systemd/src' \ ':(exclude)m4' \ ':(exclude)COPYING*' ) sed \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) *[-–] *\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C1pyright#\5 - \7#\9/' \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) *[,] *\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C2pyright#\5, \7#\9/' \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C3pyright#\5#\7/' \ -e 's/^Copyright \(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/C4pyright#\1#\3/' \ -i \ "${FILES[@]}" echo ">>> untouched Copyright lines" git grep Copyright "${FILES[@]}" echo ">>> Copyright lines with unusual extra" git grep '\<C[0-9]pyright#' "${FILES[@]}" | grep -i reserved sed \ -e 's/\<C[0-9]pyright#\([^#]*\)#\(.*\)$/Copyright (C) \1 \2/' \ -i \ "${FILES[@]}" ``` https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/298
2019-10-01 07:20:35 +00:00
* Copyright (C) 2017 Red Hat, Inc.
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
*/
#include "nm-default.h"
#include "nm-dedup-multi.h"
shared: split random and hash utils "nm-utils/nm-shared-utils.h" shall contain utility function without other dependencies. It is intended to be used by other projects as-is. nm_utils_random_bytes() requires getrandom() and a HAVE_GETRANDOM configure check. That makes it more cumbersome to re-use "nm-shared-utils.h", in cases where you don't care about nm_utils_random_bytes(). Split nm_utils_random_bytes() out to a separate file. Same for hash utils, which depend on nm_utils_random_bytes(). Also, hash utils will eventually be extended to use siphash24.
2017-10-14 11:28:20 +00:00
#include "nm-hash-utils.h"
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
#include "nm-c-list.h"
shared: split random and hash utils "nm-utils/nm-shared-utils.h" shall contain utility function without other dependencies. It is intended to be used by other projects as-is. nm_utils_random_bytes() requires getrandom() and a HAVE_GETRANDOM configure check. That makes it more cumbersome to re-use "nm-shared-utils.h", in cases where you don't care about nm_utils_random_bytes(). Split nm_utils_random_bytes() out to a separate file. Same for hash utils, which depend on nm_utils_random_bytes(). Also, hash utils will eventually be extended to use siphash24.
2017-10-14 11:28:20 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
/*****************************************************************************/
typedef struct {
/* the stack-allocated lookup entry. It has a compatible
* memory layout with NMDedupMultiEntry and NMDedupMultiHeadEntry.
*
* It is recognizable by having lst_entries_sentinel.next set to NULL.
* Contrary to the other entries, which have lst_entries.next
* always non-NULL.
* */
CList lst_entries_sentinel;
const NMDedupMultiObj * obj;
const NMDedupMultiIdxType *idx_type;
bool lookup_head;
} LookupEntry;
struct _NMDedupMultiIndex {
int ref_count;
GHashTable *idx_entries;
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
GHashTable *idx_objs;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
};
/*****************************************************************************/
static void
ASSERT_idx_type(const NMDedupMultiIdxType *idx_type)
{
nm_assert(idx_type);
#if NM_MORE_ASSERTS > 10
nm_assert(idx_type->klass);
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 09:53:08 +00:00
nm_assert(idx_type->klass->idx_obj_id_hash_update);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(idx_type->klass->idx_obj_id_equal);
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 09:53:08 +00:00
nm_assert(!!idx_type->klass->idx_obj_partition_hash_update
== !!idx_type->klass->idx_obj_partition_equal);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(idx_type->lst_idx_head.next);
#endif
}
void
nm_dedup_multi_idx_type_init(NMDedupMultiIdxType *idx_type, const NMDedupMultiIdxTypeClass *klass)
{
nm_assert(idx_type);
nm_assert(klass);
shared: use designated initializer in nm_dedup_multi_idx_type_init()
2020-07-28 08:00:23 +00:00
*idx_type = (NMDedupMultiIdxType){
.klass = klass,
.lst_idx_head = C_LIST_INIT(idx_type->lst_idx_head),
};
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
ASSERT_idx_type(idx_type);
}
/*****************************************************************************/
static NMDedupMultiEntry *
platform: add non-exclusive routes and drop route-manager Previously, we would add exclusive routes via netlink message flags NLM_F_CREATE | NLM_F_REPLACE for RTM_NEWROUTE. Similar to `ip route replace`. Using that form of RTM_NEWROUTE message, we could only add a certain route with a certain network/plen,metric triple once. That was already hugely inconvenient, because - when configuring routes, multiple (managed) interfaces may get conflicting routes (multihoming). Only one of the routes can be actually configured using `ip route replace`, so we need to track routes that are currently shadowed. - when configuring routes, we might replace externally configured routes on unmanaged interfaces. We should not interfere with such routes. That was worked around by having NMRouteManager (and NMDefaultRouteManager). NMRouteManager would keep a list of the routes which NetworkManager would like to configure, even if momentarily being unable to do so due to conflicting routes. This worked mostly well but was complicated. It involved bumping metrics to avoid conflicts for device routes, as we might require them for gateway routes. Drop that now. Instead, use the corresponding of `ip route append` to configure routes. This allows NetworkManager to confiure (almost) all routes that we care. Especially, it can configure all routes on a managed interface, without replacing/interfering with routes on other interfaces. Hence, NMRouteManager becomes obsolete. It practice it is a bit more complicated because: - when adding an IPv4 address, kernel will automatically create a device route for the subnet. We should avoid that by using the IFA_F_NOPREFIXROUTE flag for IPv4 addresses (still to-do). But as kernel may not support that flag for IPv4 addresses yet (and we don't require such a kernel yet), we still need functionality similar to nm_route_manager_ip4_route_register_device_route_purge_list(). This functionality is now handled via nm_platform_ip4_dev_route_blacklist_set(). - trying to configure an IPv6 route with a source address will be rejected by kernel as long as the address is tentative (see related bug rh#1457196). Preferably, NMDevice would keep the list of routes which should be configured, while kernel would have the list of what actually is configured. There is a feed-back loop where both affect each other (for example, when externally deleting a route, NMDevice must forget about it too). Previously, NMRouteManager would have the task of remembering all routes which we currently want to configure, but cannot due to conflicting routes. We get rid of that, because now we configure non-exclusive routes. We however still will need to remember IPv6 routes with a source address, that currently cannot be configured yet. Hence, we will need to keep track of routes that currently cannot be configured, but later may be. That is still not done yet, as NMRouteManager didn't handle this correctly either.
2017-08-14 12:18:53 +00:00
_entry_lookup_obj(const NMDedupMultiIndex * self,
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
const NMDedupMultiIdxType *idx_type,
const NMDedupMultiObj * obj)
{
const LookupEntry stack_entry = {
.obj = obj,
.idx_type = idx_type,
.lookup_head = FALSE,
};
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
ASSERT_idx_type(idx_type);
return g_hash_table_lookup(self->idx_entries, &stack_entry);
}
static NMDedupMultiHeadEntry *
platform: add non-exclusive routes and drop route-manager Previously, we would add exclusive routes via netlink message flags NLM_F_CREATE | NLM_F_REPLACE for RTM_NEWROUTE. Similar to `ip route replace`. Using that form of RTM_NEWROUTE message, we could only add a certain route with a certain network/plen,metric triple once. That was already hugely inconvenient, because - when configuring routes, multiple (managed) interfaces may get conflicting routes (multihoming). Only one of the routes can be actually configured using `ip route replace`, so we need to track routes that are currently shadowed. - when configuring routes, we might replace externally configured routes on unmanaged interfaces. We should not interfere with such routes. That was worked around by having NMRouteManager (and NMDefaultRouteManager). NMRouteManager would keep a list of the routes which NetworkManager would like to configure, even if momentarily being unable to do so due to conflicting routes. This worked mostly well but was complicated. It involved bumping metrics to avoid conflicts for device routes, as we might require them for gateway routes. Drop that now. Instead, use the corresponding of `ip route append` to configure routes. This allows NetworkManager to confiure (almost) all routes that we care. Especially, it can configure all routes on a managed interface, without replacing/interfering with routes on other interfaces. Hence, NMRouteManager becomes obsolete. It practice it is a bit more complicated because: - when adding an IPv4 address, kernel will automatically create a device route for the subnet. We should avoid that by using the IFA_F_NOPREFIXROUTE flag for IPv4 addresses (still to-do). But as kernel may not support that flag for IPv4 addresses yet (and we don't require such a kernel yet), we still need functionality similar to nm_route_manager_ip4_route_register_device_route_purge_list(). This functionality is now handled via nm_platform_ip4_dev_route_blacklist_set(). - trying to configure an IPv6 route with a source address will be rejected by kernel as long as the address is tentative (see related bug rh#1457196). Preferably, NMDevice would keep the list of routes which should be configured, while kernel would have the list of what actually is configured. There is a feed-back loop where both affect each other (for example, when externally deleting a route, NMDevice must forget about it too). Previously, NMRouteManager would have the task of remembering all routes which we currently want to configure, but cannot due to conflicting routes. We get rid of that, because now we configure non-exclusive routes. We however still will need to remember IPv6 routes with a source address, that currently cannot be configured yet. Hence, we will need to keep track of routes that currently cannot be configured, but later may be. That is still not done yet, as NMRouteManager didn't handle this correctly either.
2017-08-14 12:18:53 +00:00
_entry_lookup_head(const NMDedupMultiIndex * self,
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
const NMDedupMultiIdxType *idx_type,
const NMDedupMultiObj * obj)
{
NMDedupMultiHeadEntry *head_entry;
const LookupEntry stack_entry = {
.obj = obj,
.idx_type = idx_type,
.lookup_head = TRUE,
};
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
ASSERT_idx_type(idx_type);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (!idx_type->klass->idx_obj_partition_equal) {
if (c_list_is_empty(&idx_type->lst_idx_head))
head_entry = NULL;
else {
nm_assert(c_list_length(&idx_type->lst_idx_head) == 1);
head_entry = c_list_entry(idx_type->lst_idx_head.next, NMDedupMultiHeadEntry, lst_idx);
}
nm_assert(head_entry == g_hash_table_lookup(self->idx_entries, &stack_entry));
return head_entry;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
return g_hash_table_lookup(self->idx_entries, &stack_entry);
}
static void
_entry_unpack(const NMDedupMultiEntry * entry,
const NMDedupMultiIdxType **out_idx_type,
const NMDedupMultiObj ** out_obj,
gboolean * out_lookup_head)
{
const NMDedupMultiHeadEntry *head_entry;
const LookupEntry * lookup_entry;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(entry);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
G_STATIC_ASSERT_EXPR(G_STRUCT_OFFSET(LookupEntry, lst_entries_sentinel)
== G_STRUCT_OFFSET(NMDedupMultiEntry, lst_entries));
G_STATIC_ASSERT_EXPR(G_STRUCT_OFFSET(NMDedupMultiEntry, lst_entries)
== G_STRUCT_OFFSET(NMDedupMultiHeadEntry, lst_entries_head));
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
G_STATIC_ASSERT_EXPR(G_STRUCT_OFFSET(NMDedupMultiEntry, obj)
== G_STRUCT_OFFSET(NMDedupMultiHeadEntry, idx_type));
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
G_STATIC_ASSERT_EXPR(G_STRUCT_OFFSET(NMDedupMultiEntry, is_head)
== G_STRUCT_OFFSET(NMDedupMultiHeadEntry, is_head));
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (!entry->lst_entries.next) {
/* the entry is stack-allocated by _entry_lookup(). */
lookup_entry = (LookupEntry *) entry;
*out_obj = lookup_entry->obj;
*out_idx_type = lookup_entry->idx_type;
*out_lookup_head = lookup_entry->lookup_head;
} else if (entry->is_head) {
head_entry = (NMDedupMultiHeadEntry *) entry;
nm_assert(!c_list_is_empty(&head_entry->lst_entries_head));
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
*out_obj =
c_list_entry(head_entry->lst_entries_head.next, NMDedupMultiEntry, lst_entries)->obj;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
*out_idx_type = head_entry->idx_type;
*out_lookup_head = TRUE;
} else {
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
*out_obj = entry->obj;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
*out_idx_type = entry->head->idx_type;
*out_lookup_head = FALSE;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(NM_IN_SET(*out_lookup_head, FALSE, TRUE));
ASSERT_idx_type(*out_idx_type);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: shortcut equal operator for identical object reference in NMDedupMultiIndex And get rid of the unused obj_full_equality_allows_different_class. It's hard to grasp how to implement different object types that can compare despite having different klasses. The idea was, that stack allocated objects (used as lookup needles), are some small lightweight objects, that still compare equal to the full instance. But it's unused. Drop it.
2017-07-10 14:36:58 +00:00
/* for lookup of the head, we allow to omit object, but only
Fix typos https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/21 [thaller@redhat.com: fix generated clients/common/settings-docs.h.in file and fix wrong change in src/systemd/src/libsystemd/sd-event/sd-event.c]
2018-09-30 14:30:18 +00:00
* if the idx_type does not partition the objects. Otherwise, we
core: shortcut equal operator for identical object reference in NMDedupMultiIndex And get rid of the unused obj_full_equality_allows_different_class. It's hard to grasp how to implement different object types that can compare despite having different klasses. The idea was, that stack allocated objects (used as lookup needles), are some small lightweight objects, that still compare equal to the full instance. But it's unused. Drop it.
2017-07-10 14:36:58 +00:00
* require a obj to compare. */
nm_assert(!*out_lookup_head || (*out_obj || !(*out_idx_type)->klass->idx_obj_partition_equal));
/* lookup of the object requires always an object. */
nm_assert(*out_lookup_head || *out_obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
static guint
_dict_idx_entries_hash(const NMDedupMultiEntry *entry)
{
const NMDedupMultiIdxType *idx_type;
const NMDedupMultiObj * obj;
gboolean lookup_head;
all: refactor hashing by introducing NMHashState The privious NM_HASH_* macros directly operated on a guint value and were thus close to the actual implementation. Replace them by adding a NMHashState struct and accessors to update the hash state. This hides the implementation better and would allow us to carry more state. For example, we could switch to siphash24() transparently. For now, we still do a form basically djb2 hashing, albeit with differing start seed. Also add nm_hash_str() and nm_str_hash(): - nm_hash_str() is our own string hashing implementation - nm_str_hash() is our own string implementation, but with a GHashFunc signature, suitable to pass it to g_hash_table_new(). Also, it has this name in order to remind you of g_str_hash(), which it is replacing.
2017-10-13 12:00:22 +00:00
NMHashState h;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
_entry_unpack(entry, &idx_type, &obj, &lookup_head);
all: refactor hashing by introducing NMHashState The privious NM_HASH_* macros directly operated on a guint value and were thus close to the actual implementation. Replace them by adding a NMHashState struct and accessors to update the hash state. This hides the implementation better and would allow us to carry more state. For example, we could switch to siphash24() transparently. For now, we still do a form basically djb2 hashing, albeit with differing start seed. Also add nm_hash_str() and nm_str_hash(): - nm_hash_str() is our own string hashing implementation - nm_str_hash() is our own string implementation, but with a GHashFunc signature, suitable to pass it to g_hash_table_new(). Also, it has this name in order to remind you of g_str_hash(), which it is replacing.
2017-10-13 12:00:22 +00:00
nm_hash_init(&h, 1914869417u);
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 09:53:08 +00:00
if (idx_type->klass->idx_obj_partition_hash_update) {
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(obj);
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 09:53:08 +00:00
idx_type->klass->idx_obj_partition_hash_update(idx_type, obj, &h);
all: refactor hashing by introducing NMHashState The privious NM_HASH_* macros directly operated on a guint value and were thus close to the actual implementation. Replace them by adding a NMHashState struct and accessors to update the hash state. This hides the implementation better and would allow us to carry more state. For example, we could switch to siphash24() transparently. For now, we still do a form basically djb2 hashing, albeit with differing start seed. Also add nm_hash_str() and nm_str_hash(): - nm_hash_str() is our own string hashing implementation - nm_str_hash() is our own string implementation, but with a GHashFunc signature, suitable to pass it to g_hash_table_new(). Also, it has this name in order to remind you of g_str_hash(), which it is replacing.
2017-10-13 12:00:22 +00:00
}
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (!lookup_head)
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 09:53:08 +00:00
idx_type->klass->idx_obj_id_hash_update(idx_type, obj, &h);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
all: add helper functions for nm_hash_update*() By using a macro, we don't cast all the types to guint. Instead, we use their native types directly. Hence, we don't need nm_hash_update_uint64() nor nm_hash_update_ptr(). Also, for types smaller then guint like char, we save hashing the all zero bytes.
2017-10-16 10:38:16 +00:00
nm_hash_update_val(&h, idx_type);
all: refactor hashing by introducing NMHashState The privious NM_HASH_* macros directly operated on a guint value and were thus close to the actual implementation. Replace them by adding a NMHashState struct and accessors to update the hash state. This hides the implementation better and would allow us to carry more state. For example, we could switch to siphash24() transparently. For now, we still do a form basically djb2 hashing, albeit with differing start seed. Also add nm_hash_str() and nm_str_hash(): - nm_hash_str() is our own string hashing implementation - nm_str_hash() is our own string implementation, but with a GHashFunc signature, suitable to pass it to g_hash_table_new(). Also, it has this name in order to remind you of g_str_hash(), which it is replacing.
2017-10-13 12:00:22 +00:00
return nm_hash_complete(&h);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
static gboolean
_dict_idx_entries_equal(const NMDedupMultiEntry *entry_a, const NMDedupMultiEntry *entry_b)
{
const NMDedupMultiIdxType *idx_type_a, *idx_type_b;
const NMDedupMultiObj * obj_a, *obj_b;
gboolean lookup_head_a, lookup_head_b;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
_entry_unpack(entry_a, &idx_type_a, &obj_a, &lookup_head_a);
_entry_unpack(entry_b, &idx_type_b, &obj_b, &lookup_head_b);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (idx_type_a != idx_type_b || lookup_head_a != lookup_head_b)
return FALSE;
if (!nm_dedup_multi_idx_type_partition_equal(idx_type_a, obj_a, obj_b))
return FALSE;
if (!lookup_head_a && !nm_dedup_multi_idx_type_id_equal(idx_type_a, obj_a, obj_b))
return FALSE;
return TRUE;
}
/*****************************************************************************/
static gboolean
_add(NMDedupMultiIndex * self,
NMDedupMultiIdxType * idx_type,
const NMDedupMultiObj * obj,
NMDedupMultiEntry * entry,
NMDedupMultiIdxMode mode,
const NMDedupMultiEntry * entry_order,
NMDedupMultiHeadEntry * head_existing,
const NMDedupMultiEntry **out_entry,
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
const NMDedupMultiObj ** out_obj_old)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
NMDedupMultiHeadEntry *head_entry;
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
const NMDedupMultiObj *obj_new, *obj_old;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
gboolean add_head_entry = FALSE;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(self);
ASSERT_idx_type(idx_type);
nm_assert(obj);
nm_assert(NM_IN_SET(mode,
NM_DEDUP_MULTI_IDX_MODE_PREPEND,
NM_DEDUP_MULTI_IDX_MODE_PREPEND_FORCE,
NM_DEDUP_MULTI_IDX_MODE_APPEND,
NM_DEDUP_MULTI_IDX_MODE_APPEND_FORCE));
nm_assert(!head_existing || head_existing->idx_type == idx_type);
nm_assert(({
const NMDedupMultiHeadEntry *_h;
gboolean _ok = TRUE;
if (head_existing) {
_h = nm_dedup_multi_index_lookup_head(self, idx_type, obj);
if (head_existing == NM_DEDUP_MULTI_HEAD_ENTRY_MISSING)
_ok = (_h == NULL);
else
_ok = (_h == head_existing);
}
_ok;
}));
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (entry) {
shared: indicate changes when only reordering happens during nm_dedup_multi_index_add() The return value shall indicate whether the add-call changed anything. Reordering shall count as a change too. On the other hand, clearing the dirty flag of the entry does not count as a change.
2017-08-16 12:35:55 +00:00
gboolean changed = FALSE;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_dedup_multi_entry_set_dirty(entry, FALSE);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(!head_existing || entry->head == head_existing);
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
nm_assert(!entry_order || entry_order->head == entry->head);
nm_assert(!entry_order || c_list_contains(&entry->lst_entries, &entry_order->lst_entries));
nm_assert(!entry_order || c_list_contains(&entry_order->lst_entries, &entry->lst_entries));
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
switch (mode) {
case NM_DEDUP_MULTI_IDX_MODE_PREPEND_FORCE:
if (entry_order) {
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
if (nm_c_list_move_before((CList *) &entry_order->lst_entries, &entry->lst_entries))
shared: indicate changes when only reordering happens during nm_dedup_multi_index_add() The return value shall indicate whether the add-call changed anything. Reordering shall count as a change too. On the other hand, clearing the dirty flag of the entry does not count as a change.
2017-08-16 12:35:55 +00:00
changed = TRUE;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
} else {
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
if (nm_c_list_move_front((CList *) &entry->head->lst_entries_head,
&entry->lst_entries))
shared: indicate changes when only reordering happens during nm_dedup_multi_index_add() The return value shall indicate whether the add-call changed anything. Reordering shall count as a change too. On the other hand, clearing the dirty flag of the entry does not count as a change.
2017-08-16 12:35:55 +00:00
changed = TRUE;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
break;
case NM_DEDUP_MULTI_IDX_MODE_APPEND_FORCE:
if (entry_order) {
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
if (nm_c_list_move_after((CList *) &entry_order->lst_entries, &entry->lst_entries))
shared: indicate changes when only reordering happens during nm_dedup_multi_index_add() The return value shall indicate whether the add-call changed anything. Reordering shall count as a change too. On the other hand, clearing the dirty flag of the entry does not count as a change.
2017-08-16 12:35:55 +00:00
changed = TRUE;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
} else {
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
if (nm_c_list_move_tail((CList *) &entry->head->lst_entries_head,
&entry->lst_entries))
shared: indicate changes when only reordering happens during nm_dedup_multi_index_add() The return value shall indicate whether the add-call changed anything. Reordering shall count as a change too. On the other hand, clearing the dirty flag of the entry does not count as a change.
2017-08-16 12:35:55 +00:00
changed = TRUE;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
break;
case NM_DEDUP_MULTI_IDX_MODE_PREPEND:
case NM_DEDUP_MULTI_IDX_MODE_APPEND:
break;
};
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: shortcut equal operator for identical object reference in NMDedupMultiIndex And get rid of the unused obj_full_equality_allows_different_class. It's hard to grasp how to implement different object types that can compare despite having different klasses. The idea was, that stack allocated objects (used as lookup needles), are some small lightweight objects, that still compare equal to the full instance. But it's unused. Drop it.
2017-07-10 14:36:58 +00:00
nm_assert(obj->klass == ((const NMDedupMultiObj *) entry->obj)->klass);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (obj == entry->obj || obj->klass->obj_full_equal(obj, entry->obj)) {
NM_SET_OUT(out_entry, entry);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
NM_SET_OUT(out_obj_old, nm_dedup_multi_obj_ref(entry->obj));
shared: indicate changes when only reordering happens during nm_dedup_multi_index_add() The return value shall indicate whether the add-call changed anything. Reordering shall count as a change too. On the other hand, clearing the dirty flag of the entry does not count as a change.
2017-08-16 12:35:55 +00:00
return changed;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
obj_new = nm_dedup_multi_index_obj_intern(self, obj);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
obj_old = entry->obj;
entry->obj = obj_new;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
NM_SET_OUT(out_entry, entry);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
if (out_obj_old)
*out_obj_old = obj_old;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
else
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_dedup_multi_obj_unref(obj_old);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
return TRUE;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (idx_type->klass->idx_obj_partitionable
&& !idx_type->klass->idx_obj_partitionable(idx_type, obj)) {
/* this object cannot be partitioned by this idx_type. */
nm_assert(!head_existing || head_existing == NM_DEDUP_MULTI_HEAD_ENTRY_MISSING);
NM_SET_OUT(out_entry, NULL);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
NM_SET_OUT(out_obj_old, NULL);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
return FALSE;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
obj_new = nm_dedup_multi_index_obj_intern(self, obj);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (!head_existing)
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
head_entry = _entry_lookup_head(self, idx_type, obj_new);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
else if (head_existing == NM_DEDUP_MULTI_HEAD_ENTRY_MISSING)
head_entry = NULL;
else
head_entry = head_existing;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (!head_entry) {
head_entry = g_slice_new0(NMDedupMultiHeadEntry);
head_entry->is_head = TRUE;
head_entry->idx_type = idx_type;
c_list_init(&head_entry->lst_entries_head);
c_list_link_tail(&idx_type->lst_idx_head, &head_entry->lst_idx);
add_head_entry = TRUE;
} else
nm_assert(c_list_contains(&idx_type->lst_idx_head, &head_entry->lst_idx));
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (entry_order) {
nm_assert(!add_head_entry);
nm_assert(entry_order->head == head_entry);
nm_assert(c_list_contains(&head_entry->lst_entries_head, &entry_order->lst_entries));
nm_assert(c_list_contains(&entry_order->lst_entries, &head_entry->lst_entries_head));
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
entry = g_slice_new0(NMDedupMultiEntry);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
entry->obj = obj_new;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
entry->head = head_entry;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
switch (mode) {
case NM_DEDUP_MULTI_IDX_MODE_PREPEND:
case NM_DEDUP_MULTI_IDX_MODE_PREPEND_FORCE:
if (entry_order)
c_list_link_before((CList *) &entry_order->lst_entries, &entry->lst_entries);
else
c_list_link_front(&head_entry->lst_entries_head, &entry->lst_entries);
break;
default:
if (entry_order)
c_list_link_after((CList *) &entry_order->lst_entries, &entry->lst_entries);
else
c_list_link_tail(&head_entry->lst_entries_head, &entry->lst_entries);
break;
};
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
idx_type->len++;
head_entry->len++;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
all: require glib 2.40 RHEL 7.1 and Ubuntu 14.04 LTS both have this. https://bugzilla.gnome.org/show_bug.cgi?id=792323
2018-01-02 14:47:37 +00:00
if (add_head_entry && !g_hash_table_add(self->idx_entries, head_entry))
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert_not_reached();
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
all: require glib 2.40 RHEL 7.1 and Ubuntu 14.04 LTS both have this. https://bugzilla.gnome.org/show_bug.cgi?id=792323
2018-01-02 14:47:37 +00:00
if (!g_hash_table_add(self->idx_entries, entry))
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert_not_reached();
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
NM_SET_OUT(out_entry, entry);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
NM_SET_OUT(out_obj_old, NULL);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
return TRUE;
}
gboolean
nm_dedup_multi_index_add(NMDedupMultiIndex * self,
NMDedupMultiIdxType * idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj,
NMDedupMultiIdxMode mode,
const NMDedupMultiEntry ** out_entry,
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
/* const NMDedupMultiObj ** */ gpointer out_obj_old)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
NMDedupMultiEntry *entry;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
g_return_val_if_fail(self, FALSE);
g_return_val_if_fail(idx_type, FALSE);
g_return_val_if_fail(obj, FALSE);
g_return_val_if_fail(NM_IN_SET(mode,
NM_DEDUP_MULTI_IDX_MODE_PREPEND,
NM_DEDUP_MULTI_IDX_MODE_PREPEND_FORCE,
NM_DEDUP_MULTI_IDX_MODE_APPEND,
NM_DEDUP_MULTI_IDX_MODE_APPEND_FORCE),
FALSE);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
entry = _entry_lookup_obj(self, idx_type, obj);
return _add(self, idx_type, obj, entry, mode, NULL, NULL, out_entry, out_obj_old);
}
/* nm_dedup_multi_index_add_full:
* @self: the index instance.
* @idx_type: the index handle for storing @obj.
* @obj: the NMDedupMultiObj instance to add.
* @mode: whether to append or prepend the new item. If @entry_order is given,
* the entry will be sorted after/before, instead of appending/prepending to
* the entire list. If a comparable object is already tracked, then it may
* still be resorted by specifying one of the "FORCE" modes.
* @entry_order: if not NULL, the new entry will be sorted before or after @entry_order.
* If given, @entry_order MUST be tracked by @self, and the object it points to MUST
* be in the same partition tracked by @idx_type. That is, they must have the same
* head_entry and it means, you must ensure that @entry_order and the created/modified
* entry will share the same head.
* @entry_existing: if not NULL, it safes a hash lookup of the entry where the
* object will be placed in. You can omit this, and it will be automatically
* detected (at the expense of an additional hash lookup).
* Basically, this is the result of nm_dedup_multi_index_lookup_obj(),
Fix typos https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/21 [thaller@redhat.com: fix generated clients/common/settings-docs.h.in file and fix wrong change in src/systemd/src/libsystemd/sd-event/sd-event.c]
2018-09-30 14:30:18 +00:00
* with the peculiarity that if you know that @obj is not yet tracked,
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
* you may specify %NM_DEDUP_MULTI_ENTRY_MISSING.
* @head_existing: an optional argument to safe a lookup for the head. If specified,
Fix typos https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/21 [thaller@redhat.com: fix generated clients/common/settings-docs.h.in file and fix wrong change in src/systemd/src/libsystemd/sd-event/sd-event.c]
2018-09-30 14:30:18 +00:00
* it must be identical to nm_dedup_multi_index_lookup_head(), with the peculiarity
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
* that if the head is not yet tracked, you may specify %NM_DEDUP_MULTI_HEAD_ENTRY_MISSING
* @out_entry: if give, return the added entry. This entry may have already exists (update)
* or be newly created. If @obj is not partitionable according to @idx_type, @obj
* is not to be added and it returns %NULL.
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
* @out_obj_old: if given, return the previously contained object. It only
* returns a object, if a matching entry was tracked previously, not if a
* new entry was created. Note that when passing @out_obj_old you obtain a reference
* to the boxed object and MUST return it with nm_dedup_multi_obj_unref().
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
*
* Adds and object to the index.
*
* Return: %TRUE if anything changed, %FALSE if nothing changed.
*/
gboolean
nm_dedup_multi_index_add_full(NMDedupMultiIndex * self,
NMDedupMultiIdxType * idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj,
NMDedupMultiIdxMode mode,
const NMDedupMultiEntry * entry_order,
const NMDedupMultiEntry * entry_existing,
const NMDedupMultiHeadEntry * head_existing,
const NMDedupMultiEntry ** out_entry,
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
/* const NMDedupMultiObj ** */ gpointer out_obj_old)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
NMDedupMultiEntry *entry;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
g_return_val_if_fail(self, FALSE);
g_return_val_if_fail(idx_type, FALSE);
g_return_val_if_fail(obj, FALSE);
g_return_val_if_fail(NM_IN_SET(mode,
NM_DEDUP_MULTI_IDX_MODE_PREPEND,
NM_DEDUP_MULTI_IDX_MODE_PREPEND_FORCE,
NM_DEDUP_MULTI_IDX_MODE_APPEND,
NM_DEDUP_MULTI_IDX_MODE_APPEND_FORCE),
FALSE);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
if (entry_existing == NULL)
entry = _entry_lookup_obj(self, idx_type, obj);
else if (entry_existing == NM_DEDUP_MULTI_ENTRY_MISSING) {
nm_assert(!_entry_lookup_obj(self, idx_type, obj));
entry = NULL;
} else {
nm_assert(entry_existing == _entry_lookup_obj(self, idx_type, obj));
entry = (NMDedupMultiEntry *) entry_existing;
}
return _add(self,
idx_type,
obj,
entry,
mode,
entry_order,
(NMDedupMultiHeadEntry *) head_existing,
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
out_entry,
out_obj_old);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
/*****************************************************************************/
static void
_remove_entry(NMDedupMultiIndex *self, NMDedupMultiEntry *entry, gboolean *out_head_entry_removed)
{
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
const NMDedupMultiObj *obj;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
NMDedupMultiHeadEntry *head_entry;
NMDedupMultiIdxType * idx_type;
nm_assert(self);
nm_assert(entry);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_assert(entry->obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(entry->head);
nm_assert(!c_list_is_empty(&entry->lst_entries));
nm_assert(g_hash_table_lookup(self->idx_entries, entry) == entry);
head_entry = (NMDedupMultiHeadEntry *) entry->head;
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
obj = entry->obj;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(head_entry);
nm_assert(head_entry->len > 0);
nm_assert(g_hash_table_lookup(self->idx_entries, head_entry) == head_entry);
idx_type = (NMDedupMultiIdxType *) head_entry->idx_type;
ASSERT_idx_type(idx_type);
nm_assert(idx_type->len >= head_entry->len);
if (--head_entry->len > 0) {
nm_assert(idx_type->len > 1);
idx_type->len--;
head_entry = NULL;
}
NM_SET_OUT(out_head_entry_removed, head_entry != NULL);
if (!g_hash_table_remove(self->idx_entries, entry))
nm_assert_not_reached();
if (head_entry && !g_hash_table_remove(self->idx_entries, head_entry))
nm_assert_not_reached();
c-list: re-import latest version of c-list.h from upstream Most notably, it renames c_list_unlink_init() -> c_list_unlink() c_list_unlink() -> c_list_unlink_stale() $ sed -e 's/\<c_list_unlink\>/c_list_unlink_old/g' \ -e 's/\<c_list_unlink_init\>/c_list_unlink/g' \ -e 's/\<c_list_unlink_old\>/c_list_unlink_stale/g' \ $(git grep -l c_list_unlink -- ':(exclude)shared/nm-utils/c-list.h') \ -i
2017-11-28 10:22:01 +00:00
c_list_unlink_stale(&entry->lst_entries);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
g_slice_free(NMDedupMultiEntry, entry);
if (head_entry) {
nm_assert(c_list_is_empty(&head_entry->lst_entries_head));
c-list: re-import latest version of c-list.h from upstream Most notably, it renames c_list_unlink_init() -> c_list_unlink() c_list_unlink() -> c_list_unlink_stale() $ sed -e 's/\<c_list_unlink\>/c_list_unlink_old/g' \ -e 's/\<c_list_unlink_init\>/c_list_unlink/g' \ -e 's/\<c_list_unlink_old\>/c_list_unlink_stale/g' \ $(git grep -l c_list_unlink -- ':(exclude)shared/nm-utils/c-list.h') \ -i
2017-11-28 10:22:01 +00:00
c_list_unlink_stale(&head_entry->lst_idx);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
g_slice_free(NMDedupMultiHeadEntry, head_entry);
}
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_dedup_multi_obj_unref(obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
static guint
_remove_head(NMDedupMultiIndex * self,
NMDedupMultiHeadEntry *head_entry,
gboolean remove_all /* otherwise just dirty ones */,
gboolean mark_survivors_dirty)
{
guint n;
gboolean head_entry_removed;
CList * iter_entry, *iter_entry_safe;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(self);
nm_assert(head_entry);
nm_assert(head_entry->len > 0);
nm_assert(head_entry->len == c_list_length(&head_entry->lst_entries_head));
nm_assert(g_hash_table_lookup(self->idx_entries, head_entry) == head_entry);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
n = 0;
c_list_for_each_safe (iter_entry, iter_entry_safe, &head_entry->lst_entries_head) {
NMDedupMultiEntry *entry;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
entry = c_list_entry(iter_entry, NMDedupMultiEntry, lst_entries);
if (remove_all || entry->dirty) {
_remove_entry(self, entry, &head_entry_removed);
n++;
if (head_entry_removed)
break;
} else if (mark_survivors_dirty)
nm_dedup_multi_entry_set_dirty(entry, TRUE);
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
return n;
}
static guint
_remove_idx_entry(NMDedupMultiIndex * self,
NMDedupMultiIdxType *idx_type,
gboolean remove_all /* otherwise just dirty ones */,
gboolean mark_survivors_dirty)
{
guint n;
CList *iter_idx, *iter_idx_safe;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(self);
ASSERT_idx_type(idx_type);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
n = 0;
c_list_for_each_safe (iter_idx, iter_idx_safe, &idx_type->lst_idx_head) {
n += _remove_head(self,
c_list_entry(iter_idx, NMDedupMultiHeadEntry, lst_idx),
remove_all,
mark_survivors_dirty);
}
return n;
}
guint
nm_dedup_multi_index_remove_entry(NMDedupMultiIndex *self, gconstpointer entry)
{
g_return_val_if_fail(self, 0);
nm_assert(entry);
if (!((NMDedupMultiEntry *) entry)->is_head) {
_remove_entry(self, (NMDedupMultiEntry *) entry, NULL);
return 1;
}
return _remove_head(self, (NMDedupMultiHeadEntry *) entry, TRUE, FALSE);
}
guint
nm_dedup_multi_index_remove_obj(NMDedupMultiIndex * self,
NMDedupMultiIdxType * idx_type,
shared: return deleted object from nm_dedup_multi_index_remove_obj() For completeness of the API. remove_obj() is basically a shortcut of nm_dedup_multi_index_lookup_obj() combined with nm_dedup_multi_index_remove_entry(). As such, it is useful to return the actually deleted object. Note that the lookup needle @obj is not necessarily the same instance as the one that will be removed, it's only an instance that compares equal according to the index's equality operator.
2017-08-23 14:00:11 +00:00
/*const NMDedupMultiObj * */ gconstpointer obj,
/*const NMDedupMultiObj ** */ gconstpointer *out_obj)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
const NMDedupMultiEntry *entry;
entry = nm_dedup_multi_index_lookup_obj(self, idx_type, obj);
shared: return deleted object from nm_dedup_multi_index_remove_obj() For completeness of the API. remove_obj() is basically a shortcut of nm_dedup_multi_index_lookup_obj() combined with nm_dedup_multi_index_remove_entry(). As such, it is useful to return the actually deleted object. Note that the lookup needle @obj is not necessarily the same instance as the one that will be removed, it's only an instance that compares equal according to the index's equality operator.
2017-08-23 14:00:11 +00:00
if (!entry) {
NM_SET_OUT(out_obj, NULL);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
return 0;
shared: return deleted object from nm_dedup_multi_index_remove_obj() For completeness of the API. remove_obj() is basically a shortcut of nm_dedup_multi_index_lookup_obj() combined with nm_dedup_multi_index_remove_entry(). As such, it is useful to return the actually deleted object. Note that the lookup needle @obj is not necessarily the same instance as the one that will be removed, it's only an instance that compares equal according to the index's equality operator.
2017-08-23 14:00:11 +00:00
}
/* since we are about to remove the object, we obviously pass
* a reference to @out_obj, the caller MUST unref the object,
* if he chooses to provide @out_obj. */
NM_SET_OUT(out_obj, nm_dedup_multi_obj_ref(entry->obj));
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
_remove_entry(self, (NMDedupMultiEntry *) entry, NULL);
return 1;
}
guint
nm_dedup_multi_index_remove_head(NMDedupMultiIndex * self,
NMDedupMultiIdxType * idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj)
{
const NMDedupMultiHeadEntry *entry;
entry = nm_dedup_multi_index_lookup_head(self, idx_type, obj);
return entry ? _remove_head(self, (NMDedupMultiHeadEntry *) entry, TRUE, FALSE) : 0;
}
guint
nm_dedup_multi_index_remove_idx(NMDedupMultiIndex *self, NMDedupMultiIdxType *idx_type)
{
g_return_val_if_fail(self, 0);
g_return_val_if_fail(idx_type, 0);
return _remove_idx_entry(self, idx_type, TRUE, FALSE);
}
/*****************************************************************************/
/**
* nm_dedup_multi_index_lookup_obj:
* @self: the index cache
* @idx_type: the lookup index type
* @obj: the object to lookup. This means the match is performed
* according to NMDedupMultiIdxTypeClass's idx_obj_id_equal()
* of @idx_type.
*
* Returns: the cache entry or %NULL if the entry wasn't found.
*/
const NMDedupMultiEntry *
platform: add non-exclusive routes and drop route-manager Previously, we would add exclusive routes via netlink message flags NLM_F_CREATE | NLM_F_REPLACE for RTM_NEWROUTE. Similar to `ip route replace`. Using that form of RTM_NEWROUTE message, we could only add a certain route with a certain network/plen,metric triple once. That was already hugely inconvenient, because - when configuring routes, multiple (managed) interfaces may get conflicting routes (multihoming). Only one of the routes can be actually configured using `ip route replace`, so we need to track routes that are currently shadowed. - when configuring routes, we might replace externally configured routes on unmanaged interfaces. We should not interfere with such routes. That was worked around by having NMRouteManager (and NMDefaultRouteManager). NMRouteManager would keep a list of the routes which NetworkManager would like to configure, even if momentarily being unable to do so due to conflicting routes. This worked mostly well but was complicated. It involved bumping metrics to avoid conflicts for device routes, as we might require them for gateway routes. Drop that now. Instead, use the corresponding of `ip route append` to configure routes. This allows NetworkManager to confiure (almost) all routes that we care. Especially, it can configure all routes on a managed interface, without replacing/interfering with routes on other interfaces. Hence, NMRouteManager becomes obsolete. It practice it is a bit more complicated because: - when adding an IPv4 address, kernel will automatically create a device route for the subnet. We should avoid that by using the IFA_F_NOPREFIXROUTE flag for IPv4 addresses (still to-do). But as kernel may not support that flag for IPv4 addresses yet (and we don't require such a kernel yet), we still need functionality similar to nm_route_manager_ip4_route_register_device_route_purge_list(). This functionality is now handled via nm_platform_ip4_dev_route_blacklist_set(). - trying to configure an IPv6 route with a source address will be rejected by kernel as long as the address is tentative (see related bug rh#1457196). Preferably, NMDevice would keep the list of routes which should be configured, while kernel would have the list of what actually is configured. There is a feed-back loop where both affect each other (for example, when externally deleting a route, NMDevice must forget about it too). Previously, NMRouteManager would have the task of remembering all routes which we currently want to configure, but cannot due to conflicting routes. We get rid of that, because now we configure non-exclusive routes. We however still will need to remember IPv6 routes with a source address, that currently cannot be configured yet. Hence, we will need to keep track of routes that currently cannot be configured, but later may be. That is still not done yet, as NMRouteManager didn't handle this correctly either.
2017-08-14 12:18:53 +00:00
nm_dedup_multi_index_lookup_obj(const NMDedupMultiIndex * self,
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
const NMDedupMultiIdxType * idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj)
{
g_return_val_if_fail(self, FALSE);
g_return_val_if_fail(idx_type, FALSE);
g_return_val_if_fail(obj, FALSE);
nm_assert(idx_type && idx_type->klass);
return _entry_lookup_obj(self, idx_type, obj);
}
/**
* nm_dedup_multi_index_lookup_head:
* @self: the index cache
* @idx_type: the lookup index type
* @obj: the object to lookup, of type "const NMDedupMultiObj *".
* Depending on the idx_type, you *must* also provide a selector
* object, even when looking up the list head. That is, because
* the idx_type implementation may choose to partition the objects
* in distinct list, so you need a selector object to know which
* list head to lookup.
*
* Returns: the cache entry or %NULL if the entry wasn't found.
*/
const NMDedupMultiHeadEntry *
platform: add non-exclusive routes and drop route-manager Previously, we would add exclusive routes via netlink message flags NLM_F_CREATE | NLM_F_REPLACE for RTM_NEWROUTE. Similar to `ip route replace`. Using that form of RTM_NEWROUTE message, we could only add a certain route with a certain network/plen,metric triple once. That was already hugely inconvenient, because - when configuring routes, multiple (managed) interfaces may get conflicting routes (multihoming). Only one of the routes can be actually configured using `ip route replace`, so we need to track routes that are currently shadowed. - when configuring routes, we might replace externally configured routes on unmanaged interfaces. We should not interfere with such routes. That was worked around by having NMRouteManager (and NMDefaultRouteManager). NMRouteManager would keep a list of the routes which NetworkManager would like to configure, even if momentarily being unable to do so due to conflicting routes. This worked mostly well but was complicated. It involved bumping metrics to avoid conflicts for device routes, as we might require them for gateway routes. Drop that now. Instead, use the corresponding of `ip route append` to configure routes. This allows NetworkManager to confiure (almost) all routes that we care. Especially, it can configure all routes on a managed interface, without replacing/interfering with routes on other interfaces. Hence, NMRouteManager becomes obsolete. It practice it is a bit more complicated because: - when adding an IPv4 address, kernel will automatically create a device route for the subnet. We should avoid that by using the IFA_F_NOPREFIXROUTE flag for IPv4 addresses (still to-do). But as kernel may not support that flag for IPv4 addresses yet (and we don't require such a kernel yet), we still need functionality similar to nm_route_manager_ip4_route_register_device_route_purge_list(). This functionality is now handled via nm_platform_ip4_dev_route_blacklist_set(). - trying to configure an IPv6 route with a source address will be rejected by kernel as long as the address is tentative (see related bug rh#1457196). Preferably, NMDevice would keep the list of routes which should be configured, while kernel would have the list of what actually is configured. There is a feed-back loop where both affect each other (for example, when externally deleting a route, NMDevice must forget about it too). Previously, NMRouteManager would have the task of remembering all routes which we currently want to configure, but cannot due to conflicting routes. We get rid of that, because now we configure non-exclusive routes. We however still will need to remember IPv6 routes with a source address, that currently cannot be configured yet. Hence, we will need to keep track of routes that currently cannot be configured, but later may be. That is still not done yet, as NMRouteManager didn't handle this correctly either.
2017-08-14 12:18:53 +00:00
nm_dedup_multi_index_lookup_head(const NMDedupMultiIndex * self,
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
const NMDedupMultiIdxType * idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj)
{
g_return_val_if_fail(self, FALSE);
g_return_val_if_fail(idx_type, FALSE);
return _entry_lookup_head(self, idx_type, obj);
}
/*****************************************************************************/
void
nm_dedup_multi_index_dirty_set_head(NMDedupMultiIndex * self,
const NMDedupMultiIdxType * idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj)
{
NMDedupMultiHeadEntry *head_entry;
CList * iter_entry;
g_return_if_fail(self);
g_return_if_fail(idx_type);
head_entry = _entry_lookup_head(self, idx_type, obj);
if (!head_entry)
return;
c_list_for_each (iter_entry, &head_entry->lst_entries_head) {
NMDedupMultiEntry *entry;
entry = c_list_entry(iter_entry, NMDedupMultiEntry, lst_entries);
nm_dedup_multi_entry_set_dirty(entry, TRUE);
}
}
void
nm_dedup_multi_index_dirty_set_idx(NMDedupMultiIndex *self, const NMDedupMultiIdxType *idx_type)
{
CList *iter_idx, *iter_entry;
g_return_if_fail(self);
g_return_if_fail(idx_type);
c_list_for_each (iter_idx, &idx_type->lst_idx_head) {
NMDedupMultiHeadEntry *head_entry;
head_entry = c_list_entry(iter_idx, NMDedupMultiHeadEntry, lst_idx);
c_list_for_each (iter_entry, &head_entry->lst_entries_head) {
NMDedupMultiEntry *entry;
entry = c_list_entry(iter_entry, NMDedupMultiEntry, lst_entries);
nm_dedup_multi_entry_set_dirty(entry, TRUE);
}
}
}
/**
* nm_dedup_multi_index_dirty_remove_idx:
* @self: the index instance
* @idx_type: the index-type to select the objects.
* @mark_survivors_dirty: while the function removes all entries that are
* marked as dirty, if @set_dirty is true, the surviving objects
* will be marked dirty right away.
*
* Deletes all entries for @idx_type that are marked dirty. Only
* non-dirty objects survive. If @mark_survivors_dirty is set to TRUE, the survivors
* are marked as dirty right away.
*
* Returns: number of deleted entries.
*/
guint
nm_dedup_multi_index_dirty_remove_idx(NMDedupMultiIndex * self,
NMDedupMultiIdxType *idx_type,
gboolean mark_survivors_dirty)
{
g_return_val_if_fail(self, 0);
g_return_val_if_fail(idx_type, 0);
return _remove_idx_entry(self, idx_type, FALSE, mark_survivors_dirty);
}
/*****************************************************************************/
static guint
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
_dict_idx_objs_hash(const NMDedupMultiObj *obj)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 09:53:08 +00:00
NMHashState h;
nm_hash_init(&h, 1748638583u);
obj->klass->obj_full_hash_update(obj, &h);
return nm_hash_complete(&h);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
static gboolean
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
_dict_idx_objs_equal(const NMDedupMultiObj *obj_a, const NMDedupMultiObj *obj_b)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
core: shortcut equal operator for identical object reference in NMDedupMultiIndex And get rid of the unused obj_full_equality_allows_different_class. It's hard to grasp how to implement different object types that can compare despite having different klasses. The idea was, that stack allocated objects (used as lookup needles), are some small lightweight objects, that still compare equal to the full instance. But it's unused. Drop it.
2017-07-10 14:36:58 +00:00
return obj_a == obj_b
|| (obj_a->klass == obj_b->klass && obj_a->klass->obj_full_equal(obj_a, obj_b));
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
void
nm_dedup_multi_index_obj_release(NMDedupMultiIndex * self,
/* const NMDedupMultiObj * */ gconstpointer obj)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_assert(self);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(obj);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_assert(g_hash_table_lookup(self->idx_objs, obj) == obj);
nm_assert(((const NMDedupMultiObj *) obj)->_multi_idx == self);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
((NMDedupMultiObj *) obj)->_multi_idx = NULL;
if (!g_hash_table_remove(self->idx_objs, obj))
nm_assert_not_reached();
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
gconstpointer
nm_dedup_multi_index_obj_find(NMDedupMultiIndex * self,
/* const NMDedupMultiObj * */ gconstpointer obj)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
g_return_val_if_fail(self, NULL);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
g_return_val_if_fail(obj, NULL);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
return g_hash_table_lookup(self->idx_objs, obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
gconstpointer
nm_dedup_multi_index_obj_intern(NMDedupMultiIndex * self,
/* const NMDedupMultiObj * */ gconstpointer obj)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
const NMDedupMultiObj *obj_new = obj;
const NMDedupMultiObj *obj_old;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_assert(self);
nm_assert(obj_new);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
if (obj_new->_multi_idx == self) {
nm_assert(g_hash_table_lookup(self->idx_objs, obj_new) == obj_new);
nm_dedup_multi_obj_ref(obj_new);
return obj_new;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
obj_old = g_hash_table_lookup(self->idx_objs, obj_new);
nm_assert(obj_old != obj_new);
if (obj_old) {
nm_assert(obj_old->_multi_idx == self);
nm_dedup_multi_obj_ref(obj_old);
return obj_old;
}
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
if (nm_dedup_multi_obj_needs_clone(obj_new))
obj_new = nm_dedup_multi_obj_clone(obj_new);
else
obj_new = nm_dedup_multi_obj_ref(obj_new);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_assert(obj_new);
nm_assert(!obj_new->_multi_idx);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
all: require glib 2.40 RHEL 7.1 and Ubuntu 14.04 LTS both have this. https://bugzilla.gnome.org/show_bug.cgi?id=792323
2018-01-02 14:47:37 +00:00
if (!g_hash_table_add(self->idx_objs, (gpointer) obj_new))
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert_not_reached();
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
((NMDedupMultiObj *) obj_new)->_multi_idx = self;
return obj_new;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
shared/utils/dedup-multi: make nm_dedup_multi_obj_unref() return void This makes its prototype compatible with GDestroyNotify so that GCC 8.0 won't warn. The return value is not used anywhere and the unref() functions typically don't return any.
2018-02-07 19:23:17 +00:00
void
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_dedup_multi_obj_unref(const NMDedupMultiObj *obj)
{
if (obj) {
nm_assert(obj->_ref_count > 0);
nm_assert(obj->_ref_count != NM_OBJ_REF_COUNT_STACKINIT);
again:
if (--(((NMDedupMultiObj *) obj)->_ref_count) <= 0) {
if (obj->_multi_idx) {
/* restore the ref-count to 1 and release the object first
* from the index. Then, retry again to unref. */
((NMDedupMultiObj *) obj)->_ref_count++;
nm_dedup_multi_index_obj_release(obj->_multi_idx, obj);
nm_assert(obj->_ref_count == 1);
nm_assert(!obj->_multi_idx);
goto again;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
obj->klass->obj_destroy((NMDedupMultiObj *) obj);
}
}
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
gboolean
nm_dedup_multi_obj_needs_clone(const NMDedupMultiObj *obj)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
{
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
nm_assert(obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
if (obj->_multi_idx || obj->_ref_count == NM_OBJ_REF_COUNT_STACKINIT)
return TRUE;
if (obj->klass->obj_needs_clone && obj->klass->obj_needs_clone(obj))
return TRUE;
return FALSE;
}
const NMDedupMultiObj *
nm_dedup_multi_obj_clone(const NMDedupMultiObj *obj)
{
const NMDedupMultiObj *o;
nm_assert(obj);
o = obj->klass->obj_clone(obj);
nm_assert(o);
nm_assert(o->_ref_count == 1);
return o;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
}
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 21:34:41 +00:00
gconstpointer *
nm_dedup_multi_objs_to_array_head(const NMDedupMultiHeadEntry * head_entry,
NMDedupMultiFcnSelectPredicate predicate,
gpointer user_data,
guint * out_len)
{
gconstpointer *result;
CList * iter;
guint i;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 21:34:41 +00:00
if (!head_entry) {
NM_SET_OUT(out_len, 0);
return NULL;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 21:34:41 +00:00
result = g_new(gconstpointer, head_entry->len + 1);
i = 0;
c_list_for_each (iter, &head_entry->lst_entries_head) {
const NMDedupMultiObj *obj = c_list_entry(iter, NMDedupMultiEntry, lst_entries)->obj;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 21:34:41 +00:00
if (!predicate || predicate(obj, user_data)) {
nm_assert(i < head_entry->len);
result[i++] = obj;
}
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 21:34:41 +00:00
if (i == 0) {
g_free(result);
NM_SET_OUT(out_len, 0);
return NULL;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 21:34:41 +00:00
nm_assert(i <= head_entry->len);
NM_SET_OUT(out_len, i);
result[i++] = NULL;
return result;
}
platform: refactor nm_dedup_multi_objs_to_ptr_array_head() by moving the core functionality to "nm-dedup-multi.c". As the ref-counting mechanism now is part of "nm-dedup-multi.c", this works better and is reusable outside of platform.
2017-07-04 07:57:06 +00:00
GPtrArray *
nm_dedup_multi_objs_to_ptr_array_head(const NMDedupMultiHeadEntry * head_entry,
NMDedupMultiFcnSelectPredicate predicate,
gpointer user_data)
{
GPtrArray *result;
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 21:34:41 +00:00
CList * iter;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
platform: refactor nm_dedup_multi_objs_to_ptr_array_head() by moving the core functionality to "nm-dedup-multi.c". As the ref-counting mechanism now is part of "nm-dedup-multi.c", this works better and is reusable outside of platform.
2017-07-04 07:57:06 +00:00
if (!head_entry)
return NULL;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
platform: refactor nm_dedup_multi_objs_to_ptr_array_head() by moving the core functionality to "nm-dedup-multi.c". As the ref-counting mechanism now is part of "nm-dedup-multi.c", this works better and is reusable outside of platform.
2017-07-04 07:57:06 +00:00
result = g_ptr_array_new_full(head_entry->len, (GDestroyNotify) nm_dedup_multi_obj_unref);
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 21:34:41 +00:00
c_list_for_each (iter, &head_entry->lst_entries_head) {
const NMDedupMultiObj *obj = c_list_entry(iter, NMDedupMultiEntry, lst_entries)->obj;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
platform: refactor nm_dedup_multi_objs_to_ptr_array_head() by moving the core functionality to "nm-dedup-multi.c". As the ref-counting mechanism now is part of "nm-dedup-multi.c", this works better and is reusable outside of platform.
2017-07-04 07:57:06 +00:00
if (!predicate || predicate(obj, user_data))
g_ptr_array_add(result, (gpointer) nm_dedup_multi_obj_ref(obj));
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
platform: refactor nm_dedup_multi_objs_to_ptr_array_head() by moving the core functionality to "nm-dedup-multi.c". As the ref-counting mechanism now is part of "nm-dedup-multi.c", this works better and is reusable outside of platform.
2017-07-04 07:57:06 +00:00
if (result->len == 0) {
g_ptr_array_unref(result);
return NULL;
}
return result;
}
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
/**
* nm_dedup_multi_entry_reorder:
* @entry: the entry to reorder. It must not be NULL (and tracked in an index).
* @entry_order: (allow-none): an optional other entry. It MUST be in the same
* list as entry. If given, @entry will be ordered after/before @entry_order.
* If left at %NULL, @entry will be moved to the front/end of the list.
* @order_after: if @entry_order is given, %TRUE means to move @entry after
* @entry_order (otherwise before).
* If @entry_order is %NULL, %TRUE means to move @entry to the tail of the list
* (otherwise the beginning). Note that "tail of the list" here means that @entry
* will be linked before the head of the circular list.
*
* Returns: %TRUE, if anything was changed. Otherwise, @entry was already at the
* right place and nothing was done.
*/
gboolean
nm_dedup_multi_entry_reorder(const NMDedupMultiEntry *entry,
const NMDedupMultiEntry *entry_order,
gboolean order_after)
{
nm_assert(entry);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
if (!entry_order) {
const NMDedupMultiHeadEntry *head_entry = entry->head;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
if (order_after) {
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
if (nm_c_list_move_tail((CList *) &head_entry->lst_entries_head,
(CList *) &entry->lst_entries))
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
return TRUE;
} else {
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
if (nm_c_list_move_front((CList *) &head_entry->lst_entries_head,
(CList *) &entry->lst_entries))
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
return TRUE;
}
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
} else {
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
if (order_after) {
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
if (nm_c_list_move_after((CList *) &entry_order->lst_entries,
(CList *) &entry->lst_entries))
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
return TRUE;
} else {
all: use nm_c_list_move_*() helpers
2019-02-12 10:05:43 +00:00
if (nm_c_list_move_before((CList *) &entry_order->lst_entries,
(CList *) &entry->lst_entries))
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
return TRUE;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
}
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 08:26:17 +00:00
return FALSE;
}
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
/*****************************************************************************/
NMDedupMultiIndex *
nm_dedup_multi_index_new(void)
{
NMDedupMultiIndex *self;
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
self = g_slice_new0(NMDedupMultiIndex);
self->ref_count = 1;
self->idx_entries =
g_hash_table_new((GHashFunc) _dict_idx_entries_hash, (GEqualFunc) _dict_idx_entries_equal);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
self->idx_objs =
g_hash_table_new((GHashFunc) _dict_idx_objs_hash, (GEqualFunc) _dict_idx_objs_equal);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
return self;
}
NMDedupMultiIndex *
nm_dedup_multi_index_ref(NMDedupMultiIndex *self)
{
g_return_val_if_fail(self, NULL);
g_return_val_if_fail(self->ref_count > 0, NULL);
self->ref_count++;
return self;
}
NMDedupMultiIndex *
nm_dedup_multi_index_unref(NMDedupMultiIndex *self)
{
GHashTableIter iter;
const NMDedupMultiIdxType *idx_type;
NMDedupMultiEntry * entry;
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
const NMDedupMultiObj * obj;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
g_return_val_if_fail(self, NULL);
g_return_val_if_fail(self->ref_count > 0, NULL);
if (--self->ref_count > 0)
return NULL;
more:
g_hash_table_iter_init(&iter, self->idx_entries);
while (g_hash_table_iter_next(&iter, (gpointer *) &entry, NULL)) {
if (entry->is_head)
idx_type = ((NMDedupMultiHeadEntry *) entry)->idx_type;
else
idx_type = entry->head->idx_type;
_remove_idx_entry(self, (NMDedupMultiIdxType *) idx_type, TRUE, FALSE);
goto more;
}
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
nm_assert(g_hash_table_size(self->idx_entries) == 0);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
g_hash_table_iter_init(&iter, self->idx_objs);
while (g_hash_table_iter_next(&iter, (gpointer *) &obj, NULL)) {
nm_assert(obj->_multi_idx == self);
((NMDedupMultiObj *) obj)->_multi_idx = NULL;
}
g_hash_table_remove_all(self->idx_objs);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
g_hash_table_unref(self->idx_entries);
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 21:46:06 +00:00
g_hash_table_unref(self->idx_objs);
all: reformat all with new clang-format style Run: ./contrib/scripts/nm-code-format.sh -i ./contrib/scripts/nm-code-format.sh -i Yes, it needs to run twice because the first run doesn't yet produce the final result. Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 14:03:33 +00:00
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 20:43:21 +00:00
g_slice_free(NMDedupMultiIndex, self);
return NULL;
}
Reference in a new issue Copy permalink