2002-03-26 22:07:38 +00:00
|
|
|
/*-
|
2017-11-27 15:17:37 +00:00
|
|
|
* SPDX-License-Identifier: BSD-3-Clause
|
|
|
|
|
*
|
2002-03-26 22:07:38 +00:00
|
|
|
* Copyright (c) 2002 Poul-Henning Kamp
|
|
|
|
|
* Copyright (c) 2002 Networks Associates Technology, Inc.
|
|
|
|
|
* All rights reserved.
|
|
|
|
|
*
|
|
|
|
|
* This software was developed for the FreeBSD Project by Poul-Henning Kamp
|
|
|
|
|
* and NAI Labs, the Security Research Division of Network Associates, Inc.
|
|
|
|
|
* under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the
|
|
|
|
|
* DARPA CHATS research program.
|
|
|
|
|
*
|
|
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
|
* are met:
|
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
|
|
|
|
* 3. The names of the authors may not be used to endorse or promote
|
|
|
|
|
* products derived from this software without specific prior written
|
|
|
|
|
* permission.
|
|
|
|
|
*
|
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
|
|
|
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
|
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
|
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
|
|
|
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
|
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
|
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
|
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
|
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
|
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
|
|
|
* SUCH DAMAGE.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
LIST_HEAD(class_list_head, g_class);
|
|
|
|
|
TAILQ_HEAD(g_tailq_head, g_geom);
|
|
|
|
|
|
2003-02-07 23:08:24 +00:00
|
|
|
extern int g_collectstats;
|
Merge GEOM direct dispatch changes from the projects/camlock branch.
When safety requirements are met, it allows to avoid passing I/O requests
to GEOM g_up/g_down thread, executing them directly in the caller context.
That allows to avoid CPU bottlenecks in g_up/g_down threads, plus avoid
several context switches per I/O.
The defined now safety requirements are:
- caller should not hold any locks and should be reenterable;
- callee should not depend on GEOM dual-threaded concurency semantics;
- on the way down, if request is unmapped while callee doesn't support it,
the context should be sleepable;
- kernel thread stack usage should be below 50%.
To keep compatibility with GEOM classes not meeting above requirements
new provider and consumer flags added:
- G_CF_DIRECT_SEND -- consumer code meets caller requirements (request);
- G_CF_DIRECT_RECEIVE -- consumer code meets callee requirements (done);
- G_PF_DIRECT_SEND -- provider code meets caller requirements (done);
- G_PF_DIRECT_RECEIVE -- provider code meets callee requirements (request).
Capable GEOM class can set them, allowing direct dispatch in cases where
it is safe. If any of requirements are not met, request is queued to
g_up or g_down thread same as before.
Such GEOM classes were reviewed and updated to support direct dispatch:
CONCAT, DEV, DISK, GATE, MD, MIRROR, MULTIPATH, NOP, PART, RAID, STRIPE,
VFS, ZERO, ZFS::VDEV, ZFS::ZVOL, all classes based on g_slice KPI (LABEL,
MAP, FLASHMAP, etc).
To declare direct completion capability disk(9) KPI got new flag equivalent
to G_PF_DIRECT_SEND -- DISKFLAG_DIRECT_COMPLETION. da(4) and ada(4) disk
drivers got it set now thanks to earlier CAM locking work.
This change more then twice increases peak block storage performance on
systems with manu CPUs, together with earlier CAM locking changes reaching
more then 1 million IOPS (512 byte raw reads from 16 SATA SSDs on 4 HBAs to
256 user-level threads).
Sponsored by: iXsystems, Inc.
MFC after: 2 months
2013-10-22 08:22:19 +00:00
|
|
|
#define G_STATS_PROVIDERS 1 /* Collect I/O stats for providers */
|
|
|
|
|
#define G_STATS_CONSUMERS 2 /* Collect I/O stats for consumers */
|
|
|
|
|
|
2002-03-26 22:07:38 +00:00
|
|
|
/* geom_dump.c */
|
2003-04-02 20:41:18 +00:00
|
|
|
void g_confxml(void *, int flag);
|
GEOM: Introduce partial confxml API
Traditionally the GEOM's primary channel of information from kernel to
user-space was confxml, fetched by libgeom through kern.geom.confxml
sysctl. It is convenient and informative, representing full state of
GEOM in a single XML document. But problems start to arise on systems
with hundreds of disks, where the full confxml size reaches many
megabytes, taking significant time to first write it and then parse.
This patch introduces alternative solution, allowing to fetch much
smaller XML document, subset of the full confxml, limited to 64KB and
representing only one specified geom and optionally its parents. It
uses existing GEOM control interface, extended with new "getxml" verb.
In case of any error, such as the buffer overflow, it just transparently
falls back to traditional full confxml. This patch uses the new API in
user-space GEOM tools where it is possible.
Reviewed by: imp
MFC after: 2 month
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D34529
2022-03-12 16:49:37 +00:00
|
|
|
void g_conf_specific(struct sbuf *sb, struct g_geom **gps);
|
2019-06-19 15:36:02 +00:00
|
|
|
void g_conf_cat_escaped(struct sbuf *sb, const char *buf);
|
2013-11-27 14:25:06 +00:00
|
|
|
void g_conf_printf_escaped(struct sbuf *sb, const char *fmt, ...);
|
2003-04-02 20:41:18 +00:00
|
|
|
void g_confdot(void *, int flag);
|
|
|
|
|
void g_conftxt(void *, int flag);
|
2002-03-26 22:07:38 +00:00
|
|
|
|
|
|
|
|
/* geom_event.c */
|
|
|
|
|
void g_event_init(void);
|
|
|
|
|
void g_run_events(void);
|
2004-07-08 16:17:14 +00:00
|
|
|
void g_do_wither(void);
|
2002-03-26 22:07:38 +00:00
|
|
|
|
|
|
|
|
/* geom_subr.c */
|
2002-04-04 09:41:47 +00:00
|
|
|
extern struct class_list_head g_classes;
|
2022-01-30 03:12:43 +00:00
|
|
|
extern char *g_wait_event, *g_wait_up, *g_wait_down;
|
2013-03-24 03:15:20 +00:00
|
|
|
void g_wither_washer(void);
|
2002-03-26 22:07:38 +00:00
|
|
|
|
|
|
|
|
/* geom_io.c */
|
|
|
|
|
void g_io_init(void);
|
|
|
|
|
void g_io_schedule_down(struct thread *tp);
|
|
|
|
|
void g_io_schedule_up(struct thread *tp);
|
|
|
|
|
|
|
|
|
|
/* geom_kern.c / geom_kernsim.c */
|
|
|
|
|
void g_init(void);
|
2022-01-28 19:12:29 +00:00
|
|
|
extern struct thread *g_up_td;
|
|
|
|
|
extern struct thread *g_down_td;
|
2003-04-23 19:15:27 +00:00
|
|
|
extern int g_shutdown;
|
2013-09-24 20:05:16 +00:00
|
|
|
extern int g_notaste;
|
2003-03-24 13:37:15 +00:00
|
|
|
|
|
|
|
|
/* geom_ctl.c */
|
|
|
|
|
void g_ctl_init(void);
|