system/systemd
system/systemd
1
0
Fork 0
mirror of https://github.com/systemd/systemd synced 2024-10-01 13:55:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
systemd/coccinelle
History
Frantisek Sumsal c988ef4cf4 coccinelle: rework how we run the Coccinelle transformations 
Turns out that the original way we did things was quite broken, as it
skipped a _lot_ of code. This was because we just threw everything into
one pile and tried to spatch it, but this made Coccinelle sad, like when
man page examples redefined some of our macros, causing typedef
conflicts.

For example, with a minimal reproducer that defines a cleanup macro in
two source files, Coccinelle has no issues when spatch-ing each one
separately:

$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci main.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: main.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments
SPECIAL NAMES: adding _cleanup_free_ as a attribute

$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci
logcontrol-example.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: logcontrol-example.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments

But when you try to spatch both of them at once, Coccinelle starts
complaining and skipping the "bad" code:

$ spatch --verbose-parsing --sp-file zz-drop-braces.cocci main.c logcontrol-example.c
init_defs_builtins: /usr/lib64/coccinelle/standard.h
HANDLING: main.c logcontrol-example.c
SPECIAL NAMES: adding _cleanup_ as a attribute with arguments
SPECIAL NAMES: adding _cleanup_free_ as a attribute
remapping: _cleanup_ to an ident in macro name
ERROR-RECOV: found sync end of #define, line 44
parsing pass2: try again
ERROR-RECOV: found sync end of #define, line 44
parse error
 = File "logcontrol-example.c", line 44, column 21, charpos = 1719
  around = '__attribute__',
  whole content = #define _cleanup_(f) __attribute__((cleanup(f)))
badcount: 2
bad: #include <systemd/sd-journal.h>
bad:
BAD:!!!!! #define _cleanup_(f) __attribute__((cleanup(f)))

This was, unfortunately, hidden as it is visible only with
--verbose-parsing (or --parse-error-msg).

Another issue was how we handled includes. The original way of throwing
them into the pile of source files doesn't really work, leading up to
similar issues as above. The better way is to let Coccinelle properly
resolve all includes by telling it where to find our own include files
(basically the same thing we already do during compilation).

After fixing all this, Coccinelle now has a chance to process much more
of our code (there are still some issues in more complex macros, but
that requires further investigation). However, there's a huge downside
from all of this - doing a _proper_ code analysis is surprisingly time
and resource heavy; meaning that processing just one Coccinelle rule now
takes 15 - 30 minutes.

To make this slightly less painful, Coccinelle supports caching the
generated ASTs, which actually helps a lot - it gets the runtime of one
rule from 15 - 30 minutes down to ~1 minute. It, of course, has its own
downside - the cache is _really_ big (ATTOW the cache takes ~15 GiB).

However, even with the aggressive AST caching you're still looking at
~1 hour for one full Coccinelle run, which is a bit annoying, but I
guess that's the price of doing things _properly_ (but I'll definitely
look into ways of further optimizing this).
2023-12-25 13:52:42 +01:00
..
bool-cast.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
close-above-stdio.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
cmp.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
cond-omit-middle.cocci coccinelle: add a transformation for GNU conditionals 2023-03-18 14:23:11 +01:00
const-strlen.disabled licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
debug-logging.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
div-round-up.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
dup-fcntl.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
empty-or-dash.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
empty-or-root.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
empty-to-null.cocci coccinelle: skip the empty-to-null transformation on the macro itself 2023-01-25 11:35:06 +01:00
empty-to-root.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
enotsup.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
equals-null.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
errno-check.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
errno-wrapper.cocci tree-wide: use cocinnelle to apply _NEG_ macros 2023-08-16 12:52:56 +02:00
errno.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
exit-0.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
flags-set.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
fopen-unlocked.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
free_and_replace.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
hashmap_free.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
htonl.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
in_set.cocci coccinelle: don't try to use IN_SET() in assert_cc() 2022-05-30 18:17:11 +02:00
iovec-make.cocci coccinelle: don't run iovec-make on iovec_done{,_erase} 2023-10-25 11:16:37 +02:00
isempty.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
log-json.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
macros.h treewide: drop "RUN_" from "RUN_WITH_UMASK" 2022-12-13 14:39:30 +01:00
malloc_multiply.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
mempcpy.cocci coccinelle: automatically switch some uses of memcpy() → mempcpy() 2022-02-16 17:26:26 +01:00
memzero.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
mfree.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
mfree_return.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
no-if-assignments.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
not_in_set.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
o-ndelay.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
reallocarray.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
redundant-if.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
run-coccinelle.sh coccinelle: rework how we run the Coccinelle transformations 2023-12-25 13:52:42 +01:00
safe_close-no-if.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
safe_close.cocci tree-wide: use -EBADF for fd initialization 2022-12-19 15:00:57 +01:00
safe_closedir.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
safe_fclose.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
sd_event_source_disable_unref.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
set_ensure_put.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
strdupa.cocci alloc-util: add strdupa_safe() + strndupa_safe() and use it everywhere 2021-10-14 15:57:52 +02:00
strempty.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
strjoin.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
strjoina.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
strv_free.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
swap-two.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
synthetic-errno.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
take-fd.cocci coccinelle/take-fd: match for -EBADF instead of -1 2023-07-09 14:07:48 +09:00
take-ptr.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
timestamp-is-set.cocci coccinelle: automatically look for timestamp_is_set candidates 2022-02-22 22:47:12 +01:00
while-true.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
xsprintf.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00
zz-drop-braces.cocci licensing: add spdx to our .cocci files 2021-10-01 14:45:00 +02:00