system/freebsd-src
system/freebsd-src
1
0
Fork 0
mirror of https://github.com/freebsd/freebsd-src synced 2024-10-18 06:14:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Old PCIe implementations cannot allow a DMA transfer to cross a 4GB

Browse source 
boundary.  This was addressed several years ago by creating a parent
tag hierarchy for the root buses that set the boundary restriction
for appropriate buses and allowed child deviced to inherit it.
Somewhere along the way, this restriction was turned into a case for
marking the tag as a candidate for needing bounce buffers, instead
of just splitting the segment along the boundary line.  This flag
also causes all maps associated with this tag to be non-NULL, which
in turn causes bus_dmamap_sync() to take the slow path of function
pointer indirection to discover that there's no bouncing work to
do.  The end result is a lot of pages set aside in bounce pools
that will never be used, and a slow path for data buffers in nearly
every DMA-capable PCIe device.  For example, our workload at Netflix
was spending nearly 1% of all CPU time going through this slow path.

Fix this problem by being more selective about when to set the
COULD_BOUNCE flag.  Only set it when the boundary restriction
exists and the consumer cannot do more than a single DMA segment
at once.  This fixes the case of dynamic buffers (mbufs, bio's)
but doesn't address static buffers allocated from bus_dmamem_alloc().
That case will be addressed in the future.

For those interested, this was discovered thanks to Dtrace Flame
Graphs.

Discussed with: jhb, kib
Obtained from:	Netflix, Inc.
MFC after:	3 days
This commit is contained in:
Scott Long 2014-05-20 22:43:17 +00:00
parent 33f50a0985
commit 96d67b46df
Notes: svn2git 2020-12-20 02:59:44 +00:00 
svn path=/head/; revision=266481
1 changed files with 25 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
sys/x86/x86/busdma_bounce.c
View file

@ -172,12 +172,35 @@ bounce_bus_dma_tag_create(bus_dma_tag_t parent, bus_size_t alignment,
newtag->map_count = 0;
newtag->segments = NULL;
/*
* Bouncing might be needed if there's a filter.
* XXX Filters are likely broken as there's no way to
* guarantee that bounce pages will also satisfy the
* filter requirement.
*/
if (parent != NULL && ((newtag->common.filter != NULL) ||
((parent->common.flags & BUS_DMA_COULD_BOUNCE) != 0)))
newtag->common.flags |= BUS_DMA_COULD_BOUNCE;
if (newtag->common.lowaddr < ptoa((vm_paddr_t)Maxmem) ||
newtag->common.alignment > 1)
/*
* Bouncing might be needed if there's an upper memory
* restriction.
*/
if (newtag->common.lowaddr < ptoa((vm_paddr_t)Maxmem))
newtag->common.flags |= BUS_DMA_COULD_BOUNCE;
/*
* Bouncing might be needed if there's an alignment
* restriction that can't be satisfied by breaking up
* the segment.
* XXX Need to consider non-natural alignment.
* XXX Static allocations that tie to bus_dmamem_alloc()
* will likely pass this test and be penalized with
* the COULD_BOUNCE flag. Should probably have
* bus_dmamem_alloc() clear this flag.
*/
if ((newtag->common.nsegments <= 1) &&
(newtag->common.alignment > 1))
newtag->common.flags |= BUS_DMA_COULD_BOUNCE;
if (((newtag->common.flags & BUS_DMA_COULD_BOUNCE) != 0) &&