Commit graph

35 commits

Author SHA1 Message Date
Lexi Winter e1e636193d tests: move atf_python/sys/ into the tests package
Reviewed by:	markj
MFC after:	1 week
Pull Request:	https://github.com/freebsd/freebsd-src/pull/1174
2024-04-19 14:48:37 -04:00
Jose Luis Duran d4a5d4952e atf_python: Standardize custom sections
To be replaced with pytest's section/add_report_section.

Reviewed by:	markj
MFC after:	1 week
Pull Request:	https://github.com/freebsd/freebsd-src/pull/865
2023-10-13 15:31:30 -04:00
Jose Luis Duran 8a30ab5369 atf_python: Set verbosity for commands in a vnet
Reviewed by:	markj
MFC after:	1 week
Pull Request:	https://github.com/freebsd/freebsd-src/pull/865
2023-10-13 15:31:30 -04:00
Jose Luis Duran a1eb150ce2 atf_python: vnet: Use absolute paths within helpers
Usually tests are run in sterile environments; however, there is a
slight chance that the PATH overrides the utilities used for testing.

Pedantically use absolute paths, even inside VNETs, to avoid ambiguity.

Chiefly, jexec -> /usr/sbin/jexec, and ifconfig -> /sbin/ifconfig.

Reviewed by:	markj
MFC after:	1 week
Pull Request:	https://github.com/freebsd/freebsd-src/pull/865
2023-10-13 15:31:30 -04:00
Jose Luis Duran 7964a28ccf atf_python: Actually implement verbosity for run_cmd
Reviewed by:	markj
MFC after:	1 week
Pull Request:	https://github.com/freebsd/freebsd-src/pull/865
2023-10-13 15:31:30 -04:00
Jose Luis Duran 2e620256bd atf_python: Do not attempt to automatically delete lo0
Else `ifconfig lo0 destroy` will throw an:

    ifconfig: SIOCIFDESTROY: Invalid argument

Reviewed by:	markj
MFC after:	1 week
Pull Request:	https://github.com/freebsd/freebsd-src/pull/865
2023-10-13 15:31:30 -04:00
Kristof Provost b23dbabb7f pf: test rules evaluation in the face of multiple IPv6 fragment headers
Send an ICMPv6 echo request packet with multiple IPv6 fragment headers.
Set rules to pass all packets, except for ICMPv6 echo requests.

pf ought to drop the echo request, but doesn't because it reassembles
the packet, and then doesn't handle the second fragment header. In other
words: it fails to detect the ICMPv6 echo header.

Reported by:	Enrico Bassetti bassetti@di.uniroma1.it (NetSecurityLab @ Sapienza University of Rome)
MFC after:	instant
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2023-08-04 15:24:16 +02:00
Alexander V. Chernikov 84b41342da ipfw: add eaction tests
MFC after:	2 weeks
2023-06-15 06:36:16 +00:00
Alexander V. Chernikov 9f44a47fd0 ipfw(8): add ioctl/instruction generation tests
Differential Revision: https://reviews.freebsd.org/D40488
MFC after:	2 weeks
2023-06-13 11:55:37 +00:00
Alexander V. Chernikov 54b955f4df netlink: add support for decoding genl ops/groups in pytest
MFC after:	2 weeks
2023-06-01 10:45:29 +00:00
Alexander V. Chernikov 10b94e4064 netlink: add support for adding/deleting interface addresses
Differential Revision: https://reviews.freebsd.org/D40103
MFC after:	2 weeks
2023-05-16 19:39:13 +00:00
Alexander V. Chernikov 584ad4126c testing: improve vnet support in pytest
* Allow vnet object to be directly referenced
 (self.vnet1 vs self.vnet_map["vnet1"])
* Allow iface object to be directly reference
 (vnet.bridge vs vnet.iface_alias_map["bridge"])
* Allow arbitrary interface alias names insted of ifX
* Add wait_objects_any() method for waiting object from
 multiple vnets
* Add wait() method for indefinite sleep on vnet handlers

MFC after:	2 weeks
2023-05-16 15:00:45 +00:00
Alexander V. Chernikov d91f8db5f1 testing: rename IfattrType to IfaAttrType for consistency
All other attribute classes are named as 'type'AttrType and 'type'
 int this case is ifa (interface address).

MFC after:	2 weeks
2023-05-15 11:47:07 +00:00
Alexander V. Chernikov f0ffe1ce0f testing: add support for handling Netlink carp messages
MFC after:	2 weeks
2023-05-15 10:48:45 +00:00
Alexander V. Chernikov f3065e767d testing: add support for using custom interfaces in pytest framework.
MFC after:	2 weeks
2023-05-15 10:44:20 +00:00
Alexander V. Chernikov 04a036601e tests: fix netlink type parsing.
Netlink attribute type field is 2 bytes, not one.

MFC after:	2 weeks
2023-04-18 14:09:47 +00:00
Alexander V. Chernikov 3e5d0784b9 Testing: add framework for the kernel unit tests.
This changes intends to reduce the bar to the kernel unit-testing by
 introducing a new kernel-testing framework ("ktest") based on Netlink,
 loadable test modules and python test suite integration.

This framework provides the following features:
* Integration to the FreeBSD test suite
* Automatic test discovery
* Automatic test module loading
* Minimal boiler-plate code in both kernel and userland
* Passing any metadata to the test
* Convenient environment pre-setup using python testing framework
* Streaming messages from the kernel to the userland
* Running tests in the dedicated taskqueues
* Skipping or parametrizing tests

Differential Revision: https://reviews.freebsd.org/D39385
MFC after:	2 weeks
2023-04-14 15:47:55 +00:00
Alexander V. Chernikov 388420e613 tests: fix utils import in netlink tests
MFC after:	2 weeks
2023-04-02 10:17:37 +00:00
Alexander V. Chernikov fc2538cb7b tests: add support for parsing generic netlink families.
MFC after:	2 weeks
Differential Revision: https://reviews.freebsd.org/D39370
2023-04-01 19:33:47 +00:00
Alexander V. Chernikov fee65b7e21 tests: split netlink.py into multiple files to impove maintainability.
This diff does not contain any functional changes.
Its sole purpose is splitting netlink.py into smaller chunks.
The new code simplifies the upcoming generic netlink support
introduction.

MFC after:	2 weeks
Differential Revision: https://reviews.freebsd.org/D39365
2023-04-01 19:31:38 +00:00
Alexander V. Chernikov 90bc2120b5 tests: fix python parsing of netlink nested attributes
MFC after:	2 weeks
2023-03-13 10:17:36 +00:00
Alexander V. Chernikov c57dfd92c8 tests: add more netlink tests for neighbors/routes
Differential Revision: https://reviews.freebsd.org/D38912
MFC after:	2 weeks
2023-03-07 17:31:26 +00:00
Alexander V. Chernikov 828d3c6c4c tests: add netlink large dump buffer check
Differential Revision: https://reviews.freebsd.org/D38665
MFC after:	2 weeks
2023-02-20 10:20:54 +00:00
Alexander V. Chernikov ff8da9b2ba netlink: improve attribute decoding in python tests
MFC after:	2 weeks
2023-02-17 18:16:08 +00:00
Alexander V. Chernikov 25c2dd2f2c netlink: return optional metadata with the operation result.
Some operations like interface creation may need to return metadata
 - in this case, interface name - back to the caller if the operation
 is successful.
This change implements attaching an `NLMSGERR_ATTR_COOKIE` nla to the
operation reply message via `nlmsg_report_cookie()`.
Additionally, on successful interface creation, interface index and
 interface name are returned in the `IFLA_NEW_IFINDEX` and `IFLA_IFNAME
 TLVs, encapsulated in the `NLMSGERR_ATTR_COOKIE`.

Reviewed By: pauamma
Differential Revision: https://reviews.freebsd.org/D38283
MFC after:	1 week
2023-02-09 15:30:00 +00:00
Alexander V. Chernikov 6332ef8941 testing: handling non-root users with VNETs in pytest-based tests.
Currently isolation and resource requirements are handled directly
 by the kyua runner, based on the requirements specified by the test.
It works well for simple tests, but may cause discrepancy with tests
 doing complex pre-setups. For example, all tests that perform
 VNET setups require root access to properly function.

This change adds additional handling of the "require_user" property
 within the python testing framework. Specifically, it requests
 root access if the test class signals its root requirements and
 drops privileges to the desired user after performing the pre-setup.

Differential Revision: https://reviews.freebsd.org/D37923
MFC after:	2 weeks
2023-02-09 14:49:27 +00:00
Alexander V. Chernikov 20ea7f26e4 pytest: delete interfaces from inside the jail.
This change follows the approach used in 80fc25025f, to
 minimise the impact of the delayed interface migration.

MFC after:	2 weeks
2023-01-25 16:34:17 +00:00
Jose Luis Duran 4856aeaaed tests: Add an IPv4 loopback address of 127.0.0.1/8 to the lo0
interface by default when creating VNETSs using pytest.

Reviewed By: asomers
Differential Revision: https://reviews.freebsd.org/D38021
2023-01-11 14:34:41 +00:00
Alexander V. Chernikov c1871a3372 netlink: improve RTM_GETADDR handling.
* Allow filtering by ifa_family & ifa_index.
* Add common RTM_<NEW|DEL|GET>ADDR parser
* Add tests verifying RTM_GETADDR filtering behaviour & output
* Factor out common netlink socket test methods into NetlinkTestTemplate
* Add NLMSG_DONE message handler

Reviewed By: pauamma
Differential Revision: https://reviews.freebsd.org/D37970
2023-01-08 15:06:34 +00:00
Alexander V. Chernikov f63825ff21 testing: improve python vnet wrapper.
* Derive jail name from class name and method name, instead of just
method name. This change reduces the chances of different tests
clashing.
 Old: 'jail_test_one'. New: 'pytest:TestExampleSimplest:test_one'
* Simplify vnetX_handler() method signature by skipping obj_map (unused)
 and pipe. The latter can be accessed as the vnet property.
* Add `send_object()` method as a pair to the `wait_object` inside the
 VnetTestTemplate class.
* Add `test_id` property to the BaseTest method. Previously it was
 provided only for the VnetTestTemplate class. This change makes
 the identifier easily accessible for all users.

MFC after:	2 weeks
2022-12-29 19:59:11 +00:00
Alexander V. Chernikov 3873bdc2f2 netlink: connect netlink tests to the build
Reviewed By: ngie
Differential Revision: https://reviews.freebsd.org/D37708
2022-12-16 12:02:50 +00:00
Alexander V. Chernikov 80f03e63d6 netlink: improve interface handling
* Separate interface creation from interface modification code
* Support setting some interface attributes (ifdescr, mtu, up/down, promisc)
* Improve interaction with the cloners requiring to parse/write custom
 interface attributes
* Add bitmask-based way of checking if the attribute is present in the
message
* Don't use multipart RTM_GETLINK replies when searching for the
specific interface names
* Use ENODEV instead of ENOENT in case of failed RTM_GETLINK search
* Add python netlink test helpers
* Add some netlink interface tests

Differential Revision: https://reviews.freebsd.org/D37668
2022-12-14 19:52:35 +00:00
Alexander V. Chernikov 7064c94a02 tests: add routing tests for switching between same prefixes
Differential Revision: https://reviews.freebsd.org/D36055
MFC after:	2 weeks
2022-08-07 19:45:25 +00:00
Alexander V. Chernikov cfc9cf9baf testing: add ability to specify multi-vnet topologies in the pytest framework.
Notable amount of tests related to the packet IO require two VNET jails
 for proper testing and avoiding side effects for the host system.
Additionally, it is often required to run actions in the jails seme-sequentially
- waiting for the listener initialisation can be an example of such
  dependency.

This change extends pytest vnet framework to allow defining multi-vnet
 multi-epair topologies in declarative style, without any need to bother
 about jail or repair names. All jail creation/teardown, interface
 creation/teardown and address assignments are handled automatically.

Example:

TOPOLOGY = {
  "vnet1": {"ifaces": ["if1", "if2", "if3"]},
  "vnet2": {"ifaces": ["if1", "if2", "if3"]},
  "if1": {"prefixes6": [("2001:db8:a::1/64", "2001:db8:a::2/64")]},
  "if2": {"prefixes6": [("2001:db8:b::1/64", "2001:db8:b::2/64")]},
  "if3": {"prefixes6": [("2001:db8:c::1/64", "2001:db8:c::2/64")]},
}
def vnet2_handler(self, vnet, obj_map, pipe):
  ss = VerboseSocketServer("::", self.DEFAULT_PORT)
  pipe.send("READY")

def test_output6_base(self):
  self.wait_object(second_vnet.pipe)

The definitions above will create 2 vnets ("jail_test_output6_base",
 "jail_test_output6_base_2"), 3 epairs, attached to both first and
 second jails, set up the IP addresses for each epair, spawn another
 process for vnet2_handler and pass control to vnet2_handler and
 test_output6_base. Both processes can pass objects between each
 other using pre-created pipes.

Differential Revision: https://reviews.freebsd.org/D35708
2022-07-07 10:05:06 +00:00
Alexander V. Chernikov 8eb2bee6c0 testing: Add basic atf support to pytest.
Implementation consists of the pytest plugin implementing ATF format and
a simple C++ wrapper, which reorders the provided arguments from ATF format
to the format understandable by pytest. Each test has this wrapper specified
after the shebang. When kyua executes the test, wrapper calls pytest, which
loads atf plugin, does the work and returns the result. Additionally, a
separate python "package", `/usr/tests/atf_python` has been added to collect
code that may be useful across different tests.

Current limitations:
* Opaque metadata passing via X-Name properties. Require some fixtures to write
* `-s srcdir` parameter passed by the runner is ignored.
* No `atf-c-api(3)` or similar - relying on pytest framework & existing python libraries
* No support for `atf_tc_<get|has>_config_var()` & `atf_tc_set_md_var()`.
 Can be probably implemented with env variables & autoload fixtures

Differential Revision: https://reviews.freebsd.org/D31084
Reviewed by:	kp, ngie
2022-06-25 19:25:15 +00:00