system/freebsd-src
system/freebsd-src
1
0
Fork 0
mirror of https://github.com/freebsd/freebsd-src synced 2024-09-06 17:18:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
freebsd-src/sys/dev/ath/if_ath_rx_edma.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

1044 lines
25 KiB
C
Raw Normal View History

Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
/*-
spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch up to that fact and revert to their recommended match of BSD-2-Clause. Discussed with: pfg MFC After: 3 days Sponsored by: Netflix
2023-05-10 15:40:58 +00:00
* SPDX-License-Identifier: BSD-2-Clause
sys/dev: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts.
2017-11-27 14:52:40 +00:00
*
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
* Copyright (c) 2012 Adrian Chadd <adrian@FreeBSD.org>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer,
* without modification.
* 2. Redistributions in binary form must reproduce at minimum a disclaimer
* similar to the "NO WARRANTY" disclaimer below ("Disclaimer") and any
* redistribution must be conditioned upon including a substantially
* similar Disclaimer requirement for further binary redistribution.
*
* NO WARRANTY
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTIBILITY
* AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
* THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY,
* OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
* IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
* THE POSSIBILITY OF SUCH DAMAGES.
*/
#include <sys/cdefs.h>
/*
* Driver for the Atheros Wireless LAN controller.
*
* This software is derived from work of Atsushi Onoe; his contribution
* is greatly appreciated.
*/
#include "opt_inet.h"
#include "opt_ath.h"
/*
* This is needed for register operations which are performed
* by the driver - eg, calls to ath_hal_gettsf32().
*
* It's also required for any AH_DEBUG checks in here, eg the
* module dependencies.
*/
#include "opt_ah.h"
#include "opt_wlan.h"
#include <sys/param.h>
#include <sys/systm.h>
#include <sys/sysctl.h>
#include <sys/mbuf.h>
#include <sys/malloc.h>
#include <sys/lock.h>
#include <sys/mutex.h>
#include <sys/kernel.h>
#include <sys/socket.h>
#include <sys/sockio.h>
#include <sys/errno.h>
#include <sys/callout.h>
#include <sys/bus.h>
#include <sys/endian.h>
#include <sys/kthread.h>
#include <sys/taskqueue.h>
#include <sys/priv.h>
#include <sys/module.h>
#include <sys/ktr.h>
#include <sys/smp.h> /* for mp_ncpus */
#include <machine/bus.h>
#include <net/if.h>
The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare to this event, adding if_var.h to files that do need it. Also, include all includes that now are included due to implicit pollution via if_var.h Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-26 17:58:36 +00:00
#include <net/if_var.h>
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
#include <net/if_dl.h>
#include <net/if_media.h>
#include <net/if_types.h>
#include <net/if_arp.h>
#include <net/ethernet.h>
#include <net/if_llc.h>
#include <net80211/ieee80211_var.h>
#include <net80211/ieee80211_regdomain.h>
#ifdef IEEE80211_SUPPORT_SUPERG
#include <net80211/ieee80211_superg.h>
#endif
#ifdef IEEE80211_SUPPORT_TDMA
#include <net80211/ieee80211_tdma.h>
#endif
#include <net/bpf.h>
#ifdef INET
#include <netinet/in.h>
#include <netinet/if_ether.h>
#endif
#include <dev/ath/if_athvar.h>
#include <dev/ath/ath_hal/ah_devid.h> /* XXX for softled */
#include <dev/ath/ath_hal/ah_diagcodes.h>
#include <dev/ath/if_ath_debug.h>
#include <dev/ath/if_ath_misc.h>
#include <dev/ath/if_ath_tsf.h>
#include <dev/ath/if_ath_tx.h>
#include <dev/ath/if_ath_sysctl.h>
#include <dev/ath/if_ath_led.h>
#include <dev/ath/if_ath_keycache.h>
#include <dev/ath/if_ath_rx.h>
#include <dev/ath/if_ath_beacon.h>
#include <dev/ath/if_athdfs.h>
[ath] migrate ioctl and busdma memory operations out into separate source files. This should be a big no-op pass; and reduces the size of if_ath.c. I'm hopefully soon going to take a whack at the USB support for ath(4) and this'll require some reuse of the busdma memory code.
2015-11-24 03:42:58 +00:00
#include <dev/ath/if_ath_descdma.h>
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
#ifdef ATH_TX99_DIAG
#include <dev/ath/ath_tx99/ath_tx99.h>
#endif
#include <dev/ath/if_ath_rx_edma.h>
Add some hooks into the driver to attach, detach and record EDMA descriptor events. This is primarily for the TX EDMA and TX EDMA completion. I haven't yet tied it into the EDMA RX path or the legacy TX/RX path. Things that I don't quite like: * Make the pointer type 'void' in ath_softc and have if_ath_alq*() return a malloc'ed buffer. That would remove the need to include if_ath_alq.h in if_athvar.h. * The sysctl setup needs to be cleaned up.
2012-11-08 18:11:31 +00:00
#ifdef ATH_DEBUG_ALQ
#include <dev/ath/if_ath_alq.h>
#endif
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/*
* some general macros
*/
#define INCR(_l, _sz) (_l) ++; (_l) &= ((_sz) - 1)
#define DECR(_l, _sz) (_l) --; (_l) &= ((_sz) - 1)
MALLOC_DECLARE(M_ATHDEV);
/*
* XXX TODO:
*
* + Make sure the FIFO is correctly flushed and reinitialised
* through a reset;
* + Verify multi-descriptor frames work!
* + There's a "memory use after free" which needs to be tracked down
* and fixed ASAP. I've seen this in the legacy path too, so it
* may be a generic RX path issue.
*/
/*
* XXX shuffle the function orders so these pre-declarations aren't
* required!
*/
static int ath_edma_rxfifo_alloc(struct ath_softc *sc, HAL_RX_QUEUE qtype,
int nbufs);
static int ath_edma_rxfifo_flush(struct ath_softc *sc, HAL_RX_QUEUE qtype);
static void ath_edma_rxbuf_free(struct ath_softc *sc, struct ath_buf *bf);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
static void ath_edma_recv_proc_queue(struct ath_softc *sc,
HAL_RX_QUEUE qtype, int dosched);
static int ath_edma_recv_proc_deferred_queue(struct ath_softc *sc,
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
HAL_RX_QUEUE qtype, int dosched);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
static void
ath_edma_stoprecv(struct ath_softc *sc, int dodelay)
{
struct ath_hal *ah = sc->sc_ah;
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: called, dodelay=%d\n",
__func__, dodelay);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK(sc);
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
ath_hal_stoppcurecv(ah);
ath_hal_setrxfilter(ah, 0);
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
/*
ath: Minor style cleanups device_printf => DPRINTF and two whitespace adjustments Submitted by: Augustin Cavalier <waddlesplash@gmail.com> Obtained from: Haiku (4a88aa503ad4155a20931e263d24343043994ea9) MFC after: 1 week
2018-08-10 13:38:23 +00:00
*
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
*/
if (ath_hal_stopdmarecv(ah) == AH_TRUE)
sc->sc_rx_stopped = 1;
/*
* Give the various bus FIFOs (not EDMA descriptor FIFO)
* time to finish flushing out data.
*/
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
DELAY(3000);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Flush RX pending for each queue */
/* XXX should generic-ify this */
if (sc->sc_rxedma[HAL_RX_QUEUE_HP].m_rxpending) {
m_freem(sc->sc_rxedma[HAL_RX_QUEUE_HP].m_rxpending);
sc->sc_rxedma[HAL_RX_QUEUE_HP].m_rxpending = NULL;
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
}
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
if (sc->sc_rxedma[HAL_RX_QUEUE_LP].m_rxpending) {
m_freem(sc->sc_rxedma[HAL_RX_QUEUE_LP].m_rxpending);
sc->sc_rxedma[HAL_RX_QUEUE_LP].m_rxpending = NULL;
}
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_UNLOCK(sc);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: done\n", __func__);
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
}
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
/*
* Re-initialise the FIFO given the current buffer contents.
* Specifically, walk from head -> tail, pushing the FIFO contents
* back into the FIFO.
*/
static void
ath_edma_reinit_fifo(struct ath_softc *sc, HAL_RX_QUEUE qtype)
{
struct ath_rx_edma *re = &sc->sc_rxedma[qtype];
struct ath_buf *bf;
int i, j;
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: called\n", __func__);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK_ASSERT(sc);
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
i = re->m_fifo_head;
for (j = 0; j < re->m_fifo_depth; j++) {
bf = re->m_fifo[i];
Add some debugging and comments about what's going on when reinitialising the FIFO. I still see some corner cases where no RX occurs when it should be occuring. It's quite possible that there's a subtle race condition somewhere; or maybe I'm not programming the RX queues right. There's also no locking here yet, so any reset/configuration path state change (ie, enabling/disabling receive from the ioctl, net80211 taskqueue, etc) could quite possibly confuse things.
2012-07-10 07:45:47 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX,
Cast a bus address to a uintmax_t for a debug printf to fix the build on arm.
2012-07-11 15:04:20 +00:00
"%s: Q%d: pos=%i, addr=0x%jx\n",
Add some debugging and comments about what's going on when reinitialising the FIFO. I still see some corner cases where no RX occurs when it should be occuring. It's quite possible that there's a subtle race condition somewhere; or maybe I'm not programming the RX queues right. There's also no locking here yet, so any reset/configuration path state change (ie, enabling/disabling receive from the ioctl, net80211 taskqueue, etc) could quite possibly confuse things.
2012-07-10 07:45:47 +00:00
__func__,
qtype,
i,
Cast a bus address to a uintmax_t for a debug printf to fix the build on arm.
2012-07-11 15:04:20 +00:00
(uintmax_t)bf->bf_daddr);
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
ath_hal_putrxbuf(sc->sc_ah, bf->bf_daddr, qtype);
INCR(i, re->m_fifolen);
}
/* Ensure this worked out right */
if (i != re->m_fifo_tail) {
device_printf(sc->sc_dev, "%s: i (%d) != tail! (%d)\n",
__func__,
i,
re->m_fifo_tail);
}
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: done\n", __func__);
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
}
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/*
* Start receive.
*/
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
static int
ath_edma_startrecv(struct ath_softc *sc)
{
struct ath_hal *ah = sc->sc_ah;
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX,
"%s: called; resetted=%d, stopped=%d\n", __func__,
sc->sc_rx_resetted, sc->sc_rx_stopped);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK(sc);
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
/*
* Sanity check - are we being called whilst RX
* isn't stopped? If so, we may end up pushing
* too many entries into the RX FIFO and
* badness occurs.
*/
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Enable RX FIFO */
ath_hal_rxena(ah);
/*
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
* In theory the hardware has been initialised, right?
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
*/
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
if (sc->sc_rx_resetted == 1 || sc->sc_rx_stopped == 1) {
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX,
"%s: Re-initing HP FIFO\n", __func__);
ath_edma_reinit_fifo(sc, HAL_RX_QUEUE_HP);
DPRINTF(sc, ATH_DEBUG_EDMA_RX,
"%s: Re-initing LP FIFO\n", __func__);
ath_edma_reinit_fifo(sc, HAL_RX_QUEUE_LP);
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
sc->sc_rx_resetted = 0;
} else {
device_printf(sc->sc_dev,
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
"%s: called without resetting chip? "
"resetted=%d, stopped=%d\n",
__func__,
sc->sc_rx_resetted,
sc->sc_rx_stopped);
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
}
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Add up to m_fifolen entries in each queue */
/*
* These must occur after the above write so the FIFO buffers
* are pushed/tracked in the same order as the hardware will
* process them.
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
*
* XXX TODO: is this really necessary? We should've stopped
* the hardware already and reinitialised it, so it's a no-op.
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
*/
ath_edma_rxfifo_alloc(sc, HAL_RX_QUEUE_HP,
sc->sc_rxedma[HAL_RX_QUEUE_HP].m_fifolen);
ath_edma_rxfifo_alloc(sc, HAL_RX_QUEUE_LP,
sc->sc_rxedma[HAL_RX_QUEUE_LP].m_fifolen);
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
ath_mode_init(sc);
[ath] [ath_hal] [ath_hal_9300] Extend the start PCU receive to handle resetting ANI. One of the fun issues with scanning has been how the existing ANI values were programmed into the hardware when channels were changed. If you're on a really crappy channel and ANI has made you deaf then when you scan you continue to be deaf on all channels. This code passes in a flag to startpcureceive which in AR5416 and later is also used to enable ANI. This allows it to know if it's a normal operation or a scan operation. This fixes my situation at home where a temporary spot of a device going deaf due to interference starts scanning and .. can't hear anything until I restart. Now, this isn't the full fix - ideally: (a) all the ANI config and per-channel information would be migrated to the shared HAL stuff and enabled for all of the NICs; (b) when a station reassociates and some other error conditions (like missed beacons, NF calibration failures, etc) a knob to reset ANI parameters would likely help recovery. But hey, I'm committing bits of code again! woo! Tested: * AR9344 (2G), STA operation
2019-04-21 02:36:01 +00:00
ath_hal_startpcurecv(ah, (!! sc->sc_scanning));
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
/*
* We're now doing RX DMA!
*/
sc->sc_rx_stopped = 0;
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_UNLOCK(sc);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: ready\n", __func__);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
return (0);
}
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
static void
ath_edma_recv_sched_queue(struct ath_softc *sc, HAL_RX_QUEUE qtype,
int dosched)
{
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: called; qtype=%d, dosched=%d\n",
__func__, qtype, dosched);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
ATH_LOCK(sc);
ath_power_set_power_state(sc, HAL_PM_AWAKE);
ATH_UNLOCK(sc);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
ath_edma_recv_proc_queue(sc, qtype, dosched);
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
ATH_LOCK(sc);
ath_power_restore_power_state(sc);
ATH_UNLOCK(sc);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
/* XXX TODO: methodize */
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
taskqueue_enqueue(sc->sc_tq, &sc->sc_rxtask);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: done\n", __func__);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
}
static void
ath_edma_recv_sched(struct ath_softc *sc, int dosched)
{
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: called; dosched=%d\n",
__func__, dosched);
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
ATH_LOCK(sc);
ath_power_set_power_state(sc, HAL_PM_AWAKE);
ATH_UNLOCK(sc);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
ath_edma_recv_proc_queue(sc, HAL_RX_QUEUE_HP, dosched);
ath_edma_recv_proc_queue(sc, HAL_RX_QUEUE_LP, dosched);
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
ATH_LOCK(sc);
ath_power_restore_power_state(sc);
ATH_UNLOCK(sc);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
/* XXX TODO: methodize */
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
taskqueue_enqueue(sc->sc_tq, &sc->sc_rxtask);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: done\n", __func__);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
}
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
static void
ath_edma_recv_flush(struct ath_softc *sc)
{
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_RECV | ATH_DEBUG_EDMA_RX, "%s: called\n", __func__);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
Merge in some other features from the legacy RX path: * wrap the RX proc calls in the RX refcount; * call the DFS checking, fast frames staging and TX rescheduling if required. TODO: * figure out if I can just make "do TX rescheduling" mean "schedule TX taskqueue" ?
2012-07-14 07:56:47 +00:00
ATH_PCU_LOCK(sc);
sc->sc_rxproc_cnt++;
ATH_PCU_UNLOCK(sc);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
// XXX TODO: methodize; make it an RX stop/block
while (taskqueue_cancel(sc->sc_tq, &sc->sc_rxtask, NULL) != 0) {
taskqueue_drain(sc->sc_tq, &sc->sc_rxtask);
}
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
ATH_LOCK(sc);
ath_power_set_power_state(sc, HAL_PM_AWAKE);
ATH_UNLOCK(sc);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
/*
* Flush any active frames from FIFO -> deferred list
*/
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
ath_edma_recv_proc_queue(sc, HAL_RX_QUEUE_HP, 0);
ath_edma_recv_proc_queue(sc, HAL_RX_QUEUE_LP, 0);
Merge in some other features from the legacy RX path: * wrap the RX proc calls in the RX refcount; * call the DFS checking, fast frames staging and TX rescheduling if required. TODO: * figure out if I can just make "do TX rescheduling" mean "schedule TX taskqueue" ?
2012-07-14 07:56:47 +00:00
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
/*
* Process what's in the deferred queue
*/
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
/*
* XXX: If we read the tsf/channoise here and then pass it in,
* we could restore the power state before processing
* the deferred queue.
*/
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
ath_edma_recv_proc_deferred_queue(sc, HAL_RX_QUEUE_HP, 0);
ath_edma_recv_proc_deferred_queue(sc, HAL_RX_QUEUE_LP, 0);
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
ATH_LOCK(sc);
ath_power_restore_power_state(sc);
ATH_UNLOCK(sc);
Merge in some other features from the legacy RX path: * wrap the RX proc calls in the RX refcount; * call the DFS checking, fast frames staging and TX rescheduling if required. TODO: * figure out if I can just make "do TX rescheduling" mean "schedule TX taskqueue" ?
2012-07-14 07:56:47 +00:00
ATH_PCU_LOCK(sc);
sc->sc_rxproc_cnt--;
ATH_PCU_UNLOCK(sc);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_RECV | ATH_DEBUG_EDMA_RX, "%s: done\n", __func__);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
}
/*
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
* Process frames from the current queue into the deferred queue.
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
*/
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
static void
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
ath_edma_recv_proc_queue(struct ath_softc *sc, HAL_RX_QUEUE qtype,
int dosched)
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
{
struct ath_rx_edma *re = &sc->sc_rxedma[qtype];
struct ath_rx_status *rs;
struct ath_desc *ds;
struct ath_buf *bf;
struct mbuf *m;
struct ath_hal *ah = sc->sc_ah;
uint64_t tsf;
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
uint16_t nf;
int npkts = 0;
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
tsf = ath_hal_gettsf64(ah);
nf = ath_hal_getchannoise(ah, sc->sc_curchan);
sc->sc_stats.ast_rx_noise = nf;
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: called; qtype=%d, dosched=%d\n", __func__, qtype, dosched);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK(sc);
Fix up the EDMA RX setup path to correctly initialise and reset the RX FIFO. The original code was .. well, slightly more than incorrect. It showed up as stalled RX queues if the NIC needed to be frequently reinitialised (eg during scans.) This is inspired by work done by Matt Dillon over at the DragonflyBSD project. So: * track when EDMA RX has been stopped and when the MAC has been reset; * re-initialise the ring only after a reset; * track whether RX has been stopped/started - just for debugging now; * don't bother with the RX EOL stuff for EDMA - we don't need the interrupt at all. We also don't need to disable/enable the interrupt or start DMA - once new frames are pushed into the ring via the normal RX path, it'll just restart RX DMA on its own. Tested: * AR9380, STA mode * AR9380, AP mode * AR9485, STA mode * AR9462, STA mode
2014-09-20 01:22:17 +00:00
#if 1
if (sc->sc_rx_resetted == 1) {
/*
* XXX We shouldn't ever be scheduled if
* receive has been stopped - so complain
* loudly!
*/
device_printf(sc->sc_dev,
"%s: sc_rx_resetted=1! Bad!\n",
__func__);
ATH_RX_UNLOCK(sc);
return;
}
#endif
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
do {
bf = re->m_fifo[re->m_fifo_head];
/* This shouldn't occur! */
if (bf == NULL) {
device_printf(sc->sc_dev, "%s: Q%d: NULL bf?\n",
__func__,
qtype);
break;
}
m = bf->bf_m;
ds = bf->bf_desc;
/*
* Sync descriptor memory - this also syncs the buffer for us.
* EDMA descriptors are in cached memory.
*/
bus_dmamap_sync(sc->sc_dmat, bf->bf_dmamap,
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
rs = &bf->bf_status.ds_rxstat;
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
bf->bf_rxstatus = ath_hal_rxprocdesc(ah, ds, bf->bf_daddr,
NULL, rs);
[ath] don't debug RX EDMA descriptors that are not yet complete. Approved by: re@ (gjb)
2016-06-17 17:01:32 +00:00
if (bf->bf_rxstatus == HAL_EINPROGRESS)
break;
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
#ifdef ATH_DEBUG
if (sc->sc_debug & ATH_DEBUG_RECV_DESC)
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ath_printrxbuf(sc, bf, 0, bf->bf_rxstatus == HAL_OK);
Add some hooks into the driver to attach, detach and record EDMA descriptor events. This is primarily for the TX EDMA and TX EDMA completion. I haven't yet tied it into the EDMA RX path or the legacy TX/RX path. Things that I don't quite like: * Make the pointer type 'void' in ath_softc and have if_ath_alq*() return a malloc'ed buffer. That would remove the need to include if_ath_alq.h in if_athvar.h. * The sysctl setup needs to be cleaned up.
2012-11-08 18:11:31 +00:00
#endif /* ATH_DEBUG */
#ifdef ATH_DEBUG_ALQ
if (if_ath_alq_checkdebug(&sc->sc_alq, ATH_ALQ_EDMA_RXSTATUS))
if_ath_alq_post(&sc->sc_alq, ATH_ALQ_EDMA_RXSTATUS,
sc->sc_rx_statuslen, (char *) ds);
#endif /* ATH_DEBUG */
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/*
* Completed descriptor.
*/
DPRINTF(sc, ATH_DEBUG_EDMA_RX,
"%s: Q%d: completed!\n", __func__, qtype);
Merge in some other features from the legacy RX path: * wrap the RX proc calls in the RX refcount; * call the DFS checking, fast frames staging and TX rescheduling if required. TODO: * figure out if I can just make "do TX rescheduling" mean "schedule TX taskqueue" ?
2012-07-14 07:56:47 +00:00
npkts++;
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
/*
* We've been synced already, so unmap.
*/
bus_dmamap_unload(sc->sc_dmat, bf->bf_dmamap);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/*
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
* Remove the FIFO entry and place it on the completion
* queue.
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
*/
re->m_fifo[re->m_fifo_head] = NULL;
Use a per-RX-queue deferred list, rather than a single deferred list for both queues. Since ath_rx_pkt() does multi-mbuf frame recombining based on the RX queue, this needs to occur. Tested: * AR9380 (XB112), hostap mode
2013-04-16 20:21:02 +00:00
TAILQ_INSERT_TAIL(&sc->sc_rx_rxlist[qtype], bf, bf_list);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
/* Bump the descriptor FIFO stats */
INCR(re->m_fifo_head, re->m_fifolen);
re->m_fifo_depth--;
/* XXX check it doesn't fall below 0 */
} while (re->m_fifo_depth > 0);
/* Append some more fresh frames to the FIFO */
if (dosched)
ath_edma_rxfifo_alloc(sc, qtype, re->m_fifolen);
ATH_RX_UNLOCK(sc);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
/* rx signal state monitoring */
ath_hal_rxmonitor(ah, &sc->sc_halstats, sc->sc_curchan);
ATH_KTR(sc, ATH_KTR_INTERRUPTS, 1,
"ath edma rx proc: npkts=%d\n",
npkts);
return;
}
/*
* Flush the deferred queue.
*
* This destructively flushes the deferred queue - it doesn't
* call the wireless stack on each mbuf.
*/
static void
ath_edma_flush_deferred_queue(struct ath_softc *sc)
{
Correctly remove entries from the relevant receive ath_buf list before freeing them. The current code would walk the list and call the buffer free, which didn't remove it from any lists before pushing it back on the free list. Tested: AR9485, STA mode Noticed by: dillon@apollo.dragonflybsd.org
2014-01-06 03:48:32 +00:00
struct ath_buf *bf;
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
ATH_RX_LOCK_ASSERT(sc);
Use a per-RX-queue deferred list, rather than a single deferred list for both queues. Since ath_rx_pkt() does multi-mbuf frame recombining based on the RX queue, this needs to occur. Tested: * AR9380 (XB112), hostap mode
2013-04-16 20:21:02 +00:00
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
/* Free in one set, inside the lock */
Correctly remove entries from the relevant receive ath_buf list before freeing them. The current code would walk the list and call the buffer free, which didn't remove it from any lists before pushing it back on the free list. Tested: AR9485, STA mode Noticed by: dillon@apollo.dragonflybsd.org
2014-01-06 03:48:32 +00:00
while (! TAILQ_EMPTY(&sc->sc_rx_rxlist[HAL_RX_QUEUE_LP])) {
bf = TAILQ_FIRST(&sc->sc_rx_rxlist[HAL_RX_QUEUE_LP]);
TAILQ_REMOVE(&sc->sc_rx_rxlist[HAL_RX_QUEUE_LP], bf, bf_list);
Use a per-RX-queue deferred list, rather than a single deferred list for both queues. Since ath_rx_pkt() does multi-mbuf frame recombining based on the RX queue, this needs to occur. Tested: * AR9380 (XB112), hostap mode
2013-04-16 20:21:02 +00:00
/* Free the buffer/mbuf */
ath_edma_rxbuf_free(sc, bf);
}
Correctly remove entries from the relevant receive ath_buf list before freeing them. The current code would walk the list and call the buffer free, which didn't remove it from any lists before pushing it back on the free list. Tested: AR9485, STA mode Noticed by: dillon@apollo.dragonflybsd.org
2014-01-06 03:48:32 +00:00
while (! TAILQ_EMPTY(&sc->sc_rx_rxlist[HAL_RX_QUEUE_HP])) {
bf = TAILQ_FIRST(&sc->sc_rx_rxlist[HAL_RX_QUEUE_HP]);
TAILQ_REMOVE(&sc->sc_rx_rxlist[HAL_RX_QUEUE_HP], bf, bf_list);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
/* Free the buffer/mbuf */
ath_edma_rxbuf_free(sc, bf);
}
}
static int
ath_edma_recv_proc_deferred_queue(struct ath_softc *sc, HAL_RX_QUEUE qtype,
int dosched)
{
int ngood = 0;
uint64_t tsf;
struct ath_buf *bf, *next;
struct ath_rx_status *rs;
int16_t nf;
ath_bufhead rxlist;
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
struct mbuf *m;
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
TAILQ_INIT(&rxlist);
nf = ath_hal_getchannoise(sc->sc_ah, sc->sc_curchan);
/*
* XXX TODO: the NF/TSF should be stamped on the bufs themselves,
* otherwise we may end up adding in the wrong values if this
* is delayed too far..
*/
tsf = ath_hal_gettsf64(sc->sc_ah);
/* Copy the list over */
ATH_RX_LOCK(sc);
Use a per-RX-queue deferred list, rather than a single deferred list for both queues. Since ath_rx_pkt() does multi-mbuf frame recombining based on the RX queue, this needs to occur. Tested: * AR9380 (XB112), hostap mode
2013-04-16 20:21:02 +00:00
TAILQ_CONCAT(&rxlist, &sc->sc_rx_rxlist[qtype], bf_list);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
ATH_RX_UNLOCK(sc);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
/* Handle the completed descriptors */
Correctly remove entries from the relevant receive ath_buf list before freeing them. The current code would walk the list and call the buffer free, which didn't remove it from any lists before pushing it back on the free list. Tested: AR9485, STA mode Noticed by: dillon@apollo.dragonflybsd.org
2014-01-06 03:48:32 +00:00
/*
* XXX is this SAFE call needed? The ath_buf entries
* aren't modified by ath_rx_pkt, right?
*/
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
TAILQ_FOREACH_SAFE(bf, &rxlist, bf_list, next) {
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/*
* Skip the RX descriptor status - start at the data offset
*/
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
m_adj(bf->bf_m, sc->sc_rx_statuslen);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Handle the frame */
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
Make sure that 'rs' is pointing to the correct RX status.
2012-07-14 05:53:03 +00:00
rs = &bf->bf_status.ds_rxstat;
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
m = bf->bf_m;
bf->bf_m = NULL;
if (ath_rx_pkt(sc, rs, bf->bf_rxstatus, tsf, nf, qtype, bf, m))
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
ngood++;
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
}
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
if (ngood) {
sc->sc_lastrx = tsf;
}
ATH_KTR(sc, ATH_KTR_INTERRUPTS, 1,
"ath edma rx deferred proc: ngood=%d\n",
ngood);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
/* Free in one set, inside the lock */
ATH_RX_LOCK(sc);
Correctly remove entries from the relevant receive ath_buf list before freeing them. The current code would walk the list and call the buffer free, which didn't remove it from any lists before pushing it back on the free list. Tested: AR9485, STA mode Noticed by: dillon@apollo.dragonflybsd.org
2014-01-06 03:48:32 +00:00
while (! TAILQ_EMPTY(&rxlist)) {
bf = TAILQ_FIRST(&rxlist);
TAILQ_REMOVE(&rxlist, bf, bf_list);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Free the buffer/mbuf */
ath_edma_rxbuf_free(sc, bf);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
}
ATH_RX_UNLOCK(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
return (ngood);
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
}
static void
ath_edma_recv_tasklet(void *arg, int npending)
{
struct ath_softc *sc = (struct ath_softc *) arg;
Fix build breakage when one isn't building with IEEE80211_SUPPORT_SUPERG. Noticed by: mav
2012-07-14 12:15:20 +00:00
#ifdef IEEE80211_SUPPORT_SUPERG
Replay r286410. Change KPI of how device drivers that provide wireless connectivity interact with the net80211 stack. Historical background: originally wireless devices created an interface, just like Ethernet devices do. Name of an interface matched the name of the driver that created. Later, wlan(4) layer was introduced, and the wlanX interfaces become the actual interface, leaving original ones as "a parent interface" of wlanX. Kernelwise, the KPI between net80211 layer and a driver became a mix of methods that pass a pointer to struct ifnet as identifier and methods that pass pointer to struct ieee80211com. From user point of view, the parent interface just hangs on in the ifconfig list, and user can't do anything useful with it. Now, the struct ifnet goes away. The struct ieee80211com is the only KPI between a device driver and net80211. Details: - The struct ieee80211com is embedded into drivers softc. - Packets are sent via new ic_transmit method, which is very much like the previous if_transmit. - Bringing parent up/down is done via new ic_parent method, which notifies driver about any changes: number of wlan(4) interfaces, number of them in promisc or allmulti state. - Device specific ioctls (if any) are received on new ic_ioctl method. - Packets/errors accounting are done by the stack. In certain cases, when driver experiences errors and can not attribute them to any specific interface, driver updates ic_oerrors or ic_ierrors counters. Details on interface configuration with new world order: - A sequence of commands needed to bring up wireless DOESN"T change. - /etc/rc.conf parameters DON'T change. - List of devices that can be used to create wlan(4) interfaces is now provided by net.wlan.devices sysctl. Most drivers in this change were converted by me, except of wpi(4), that was done by Andriy Voskoboinyk. Big thanks to Kevin Lo for testing changes to at least 8 drivers. Thanks to pluknet@, Oliver Hartmann, Olivier Cochard, gjb@, mmoll@, op@ and lev@, who also participated in testing. Reviewed by: adrian Sponsored by: Netflix Sponsored by: Nginx, Inc.
2015-08-27 08:56:39 +00:00
struct ieee80211com *ic = &sc->sc_ic;
Fix build breakage when one isn't building with IEEE80211_SUPPORT_SUPERG. Noticed by: mav
2012-07-14 12:15:20 +00:00
#endif
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: called; npending=%d\n",
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
__func__,
npending);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
ATH_PCU_LOCK(sc);
if (sc->sc_inreset_cnt > 0) {
device_printf(sc->sc_dev, "%s: sc_inreset_cnt > 0; skipping\n",
__func__);
ATH_PCU_UNLOCK(sc);
return;
}
Merge in some other features from the legacy RX path: * wrap the RX proc calls in the RX refcount; * call the DFS checking, fast frames staging and TX rescheduling if required. TODO: * figure out if I can just make "do TX rescheduling" mean "schedule TX taskqueue" ?
2012-07-14 07:56:47 +00:00
sc->sc_rxproc_cnt++;
Add/fix EDMA RX behaviour. * For now, kickpcu should hopefully just do nothing - the PCU doesn't need 'kicking' for Osprey and later NICs. The PCU will just restart once the next FIFO entry is pushed in. * Teach "proc" about "dosched", so it can be used to just flush the FIFO contents without adding new FIFO entries. * .. and now, implement the RX "flush" routine. * Re-initialise the FIFO contents if the FIFO is empty (the DP is NULL.) When PCU RX is disabled (ie, writing RX_D to the RX configuration register) then the FIFO will be completely emptied. If the software FIFO is full, then no further descriptors are pushed into the FIFO and things stall. This all requires much, much more thorough stress testing.
2012-07-10 06:05:42 +00:00
ATH_PCU_UNLOCK(sc);
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
ATH_LOCK(sc);
ath_power_set_power_state(sc, HAL_PM_AWAKE);
ATH_UNLOCK(sc);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
ath_edma_recv_proc_deferred_queue(sc, HAL_RX_QUEUE_HP, 1);
ath_edma_recv_proc_deferred_queue(sc, HAL_RX_QUEUE_LP, 1);
Bring over some initial power save management support, reset path fixes and beacon programming / debugging into the ath(4) driver. The basic power save tracking: * Add some new code to track the current desired powersave state; and * Add some reference count tracking so we know when the NIC is awake; then * Add code in all the points where we're about to touch the hardware and push it to force-wake. Then, how things are moved into power save: * Only move into network-sleep during a RUN->SLEEP transition; * Force wake the hardware up everywhere that we're about to touch the hardware. The net80211 stack takes care of doing RUN<->SLEEP<->(other) state transitions so we don't have to do it in the driver. Next, when to wake things up: * In short - everywhere we touch the hardware. * The hardware will take care of staying awake if things are queued in the transmit queue(s); it'll then transit down to sleep if there's nothing left. This way we don't have to track the software / hardware transmit queue(s) and keep the hardware awake for those. Then, some transmit path fixes that aren't related but useful: * Force EAPOL frames to go out at the lowest rate. This improves reliability during the encryption handshake after 802.11 negotiation. Next, some reset path fixes! * Fix the overlap between reset and transmit pause so we don't transmit frames during a reset. * Some noisy environments will end up taking a lot longer to reset than normal, so extend the reset period and drop the raise the reset interval to be more realistic and give the hardware some time to finish calibration. * Skip calibration during the reset path. Tsk! Then, beacon fixes in station mode! * Add a _lot_ more debugging in the station beacon reset path. This is all quite fluid right now. * Modify the STA beacon programming code to try and take the TU gap between desired TSF and the target TU into account. (Lifted from QCA.) Tested: * AR5210 * AR5211 * AR5212 * AR5413 * AR5416 * AR9280 * AR9285 TODO: * More AP, IBSS, mesh, TDMA testing * Thorough AR9380 and later testing! * AR9160 and AR9287 testing Obtained from: QCA
2014-04-30 02:19:41 +00:00
/*
* XXX: If we read the tsf/channoise here and then pass it in,
* we could restore the power state before processing
* the deferred queue.
*/
ATH_LOCK(sc);
ath_power_restore_power_state(sc);
ATH_UNLOCK(sc);
Merge in some other features from the legacy RX path: * wrap the RX proc calls in the RX refcount; * call the DFS checking, fast frames staging and TX rescheduling if required. TODO: * figure out if I can just make "do TX rescheduling" mean "schedule TX taskqueue" ?
2012-07-14 07:56:47 +00:00
#ifdef IEEE80211_SUPPORT_SUPERG
Replay r286410. Change KPI of how device drivers that provide wireless connectivity interact with the net80211 stack. Historical background: originally wireless devices created an interface, just like Ethernet devices do. Name of an interface matched the name of the driver that created. Later, wlan(4) layer was introduced, and the wlanX interfaces become the actual interface, leaving original ones as "a parent interface" of wlanX. Kernelwise, the KPI between net80211 layer and a driver became a mix of methods that pass a pointer to struct ifnet as identifier and methods that pass pointer to struct ieee80211com. From user point of view, the parent interface just hangs on in the ifconfig list, and user can't do anything useful with it. Now, the struct ifnet goes away. The struct ieee80211com is the only KPI between a device driver and net80211. Details: - The struct ieee80211com is embedded into drivers softc. - Packets are sent via new ic_transmit method, which is very much like the previous if_transmit. - Bringing parent up/down is done via new ic_parent method, which notifies driver about any changes: number of wlan(4) interfaces, number of them in promisc or allmulti state. - Device specific ioctls (if any) are received on new ic_ioctl method. - Packets/errors accounting are done by the stack. In certain cases, when driver experiences errors and can not attribute them to any specific interface, driver updates ic_oerrors or ic_ierrors counters. Details on interface configuration with new world order: - A sequence of commands needed to bring up wireless DOESN"T change. - /etc/rc.conf parameters DON'T change. - List of devices that can be used to create wlan(4) interfaces is now provided by net.wlan.devices sysctl. Most drivers in this change were converted by me, except of wpi(4), that was done by Andriy Voskoboinyk. Big thanks to Kevin Lo for testing changes to at least 8 drivers. Thanks to pluknet@, Oliver Hartmann, Olivier Cochard, gjb@, mmoll@, op@ and lev@, who also participated in testing. Reviewed by: adrian Sponsored by: Netflix Sponsored by: Nginx, Inc.
2015-08-27 08:56:39 +00:00
ieee80211_ff_age_all(ic, 100);
Merge in some other features from the legacy RX path: * wrap the RX proc calls in the RX refcount; * call the DFS checking, fast frames staging and TX rescheduling if required. TODO: * figure out if I can just make "do TX rescheduling" mean "schedule TX taskqueue" ?
2012-07-14 07:56:47 +00:00
#endif
if (ath_dfs_tasklet_needed(sc, sc->sc_curchan))
taskqueue_enqueue(sc->sc_tq, &sc->sc_dfstask);
ATH_PCU_LOCK(sc);
sc->sc_rxproc_cnt--;
ATH_PCU_UNLOCK(sc);
[ath] Hopefully recover better-er upon RX restart on AR9380. This is all very long-standing bug stuff that is touchy and still poorly documented. Ok, here goes. The basic bug: * deleting a VAP causes the RX path (and TX path too) to be restarted without a full chip reset, which causes RX hangs on the AR9380 and later. (ie, the ones with the newer DMA engine.) The basic fix: * do an RX flush when stopping RX in ath_vap_delete() to match what happens when RX is stopped elsewhere. This ensures any pending frames are completed and we restart at the right spot; it also ensures we don't push new RX buffers into the hardware if we're stopping receive. The other issues I found: * Don't bother checking the RX packet ring in the deferred read taskqueue; that's specifically supposed to be for completing frames rather than just yanking them off the receive ring. * Cancel/drain any pending deferred read taskqueue. This isn't done inside any locks so we should be super careful here. This stops the hardware being reprogrammed at the same time in another thread/CPU whilst we're stopping RX. * .. (yes, this should be better serialised, but that's for another day. maybe.) * Add more debugging to trace what's going on here. And the fun bit: * Reinitialise the RX FIFO ONLY if we've been reset or stopped, rather than just reset. I noticed that after all the above was done I was STILL seeing RXEOL. RXEOL isn't enabled on the AR9380 so I'd only see it if I was sending TX frames (ie a ping where it'd be transmitted but never received) so I was not being spammed by RXEOL. So, as long as stuff is stopped, restart it. This seems to be doing the right thing in both AP and STA modes. What I should do next, if I ever get time: * as I said above, serialise the receive stop/start to include taskqueues * monitor RXEOL on the AR9380 and I keep seeing it spammed / lockups, just go do a full chip reset to get things back on track. It sucks, but it is better than nothing. Tested: * AR9380 AP/STA mode, adding/deleting a hostap VAP to trigger the TX/RX queue stop/start; whilst also running an iperf through it. Lots of times. Lots. Of.. Times.
2020-05-21 04:35:12 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: called; done!\n", __func__);
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
}
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/*
* Allocate an RX mbuf for the given ath_buf and initialise
* it for EDMA.
*
* + Allocate a 4KB mbuf;
* + Setup the DMA map for the given buffer;
* + Return that.
*/
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
static int
ath_edma_rxbuf_init(struct ath_softc *sc, struct ath_buf *bf)
{
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
struct mbuf *m;
int error;
int len;
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK_ASSERT(sc);
Mechanically substitute flags from historic mbuf allocator with malloc(9) flags in sys/dev.
2012-12-04 09:32:43 +00:00
m = m_getm(NULL, sc->sc_edma_bufsize, M_NOWAIT, MT_DATA);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
if (! m)
return (ENOBUFS); /* XXX ?*/
/* XXX warn/enforce alignment */
len = m->m_ext.ext_size;
#if 0
device_printf(sc->sc_dev, "%s: called: m=%p, size=%d, mtod=%p\n",
__func__,
m,
len,
mtod(m, char *));
#endif
m->m_pkthdr.len = m->m_len = m->m_ext.ext_size;
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
/*
* Populate ath_buf fields.
*/
bf->bf_desc = mtod(m, struct ath_desc *);
bf->bf_lastds = bf->bf_desc; /* XXX only really for TX? */
bf->bf_m = m;
/*
* Zero the descriptor and ensure it makes it out to the
* bounce buffer if one is required.
*
* XXX PREWRITE will copy the whole buffer; we only needed it
* to sync the first 32 DWORDS. Oh well.
*/
memset(bf->bf_desc, '\0', sc->sc_rx_statuslen);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/*
* Create DMA mapping.
*/
error = bus_dmamap_load_mbuf_sg(sc->sc_dmat,
bf->bf_dmamap, m, bf->bf_segs, &bf->bf_nseg, BUS_DMA_NOWAIT);
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
if (error != 0) {
device_printf(sc->sc_dev, "%s: failed; error=%d\n",
__func__,
error);
m_freem(m);
return (error);
}
/*
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
* Set daddr to the physical mapping page.
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
*/
bf->bf_daddr = bf->bf_segs[0].ds_addr;
/*
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
* Prepare for the upcoming read.
*
* We need to both sync some data into the buffer (the zero'ed
* descriptor payload) and also prepare for the read that's going
* to occur.
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
*/
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
bus_dmamap_sync(sc->sc_dmat, bf->bf_dmamap,
BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Finish! */
return (0);
}
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
/*
* Allocate a RX buffer.
*/
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
static struct ath_buf *
ath_edma_rxbuf_alloc(struct ath_softc *sc)
{
struct ath_buf *bf;
int error;
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK_ASSERT(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Allocate buffer */
bf = TAILQ_FIRST(&sc->sc_rxbuf);
/* XXX shouldn't happen upon startup? */
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
if (bf == NULL) {
ath: Minor style cleanups device_printf => DPRINTF and two whitespace adjustments Submitted by: Augustin Cavalier <waddlesplash@gmail.com> Obtained from: Haiku (4a88aa503ad4155a20931e263d24343043994ea9) MFC after: 1 week
2018-08-10 13:38:23 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: nothing on rxbuf?!\n",
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
__func__);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
return (NULL);
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
}
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Remove it from the free list */
TAILQ_REMOVE(&sc->sc_rxbuf, bf, bf_list);
/* Assign RX mbuf to it */
error = ath_edma_rxbuf_init(sc, bf);
if (error != 0) {
device_printf(sc->sc_dev,
"%s: bf=%p, rxbuf alloc failed! error=%d\n",
__func__,
bf,
error);
TAILQ_INSERT_TAIL(&sc->sc_rxbuf, bf, bf_list);
return (NULL);
}
return (bf);
}
static void
ath_edma_rxbuf_free(struct ath_softc *sc, struct ath_buf *bf)
{
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK_ASSERT(sc);
Only unmap the RX mbuf DMA map if there's a buffer here. The normal RX path (ath_rx_pkt()) will sync and unmap the buffer before passing it up the stack. We only need to do this if we're flushing the FIFO during reset/shutdown.
2013-04-01 20:11:19 +00:00
/*
* Only unload the frame if we haven't consumed
* the mbuf via ath_rx_pkt().
*/
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
if (bf->bf_m) {
Only unmap the RX mbuf DMA map if there's a buffer here. The normal RX path (ath_rx_pkt()) will sync and unmap the buffer before passing it up the stack. We only need to do this if we're flushing the FIFO during reset/shutdown.
2013-04-01 20:11:19 +00:00
bus_dmamap_unload(sc->sc_dmat, bf->bf_dmamap);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
m_freem(bf->bf_m);
bf->bf_m = NULL;
}
/* XXX lock? */
TAILQ_INSERT_TAIL(&sc->sc_rxbuf, bf, bf_list);
}
/*
* Allocate up to 'n' entries and push them onto the hardware FIFO.
*
* Return how many entries were successfully pushed onto the
* FIFO.
*/
static int
ath_edma_rxfifo_alloc(struct ath_softc *sc, HAL_RX_QUEUE qtype, int nbufs)
{
struct ath_rx_edma *re = &sc->sc_rxedma[qtype];
struct ath_buf *bf;
int i;
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK_ASSERT(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/*
* Allocate buffers until the FIFO is full or nbufs is reached.
*/
for (i = 0; i < nbufs && re->m_fifo_depth < re->m_fifolen; i++) {
/* Ensure the FIFO is already blank, complain loudly! */
if (re->m_fifo[re->m_fifo_tail] != NULL) {
device_printf(sc->sc_dev,
"%s: Q%d: fifo[%d] != NULL (%p)\n",
__func__,
qtype,
re->m_fifo_tail,
re->m_fifo[re->m_fifo_tail]);
/* Free the slot */
ath_edma_rxbuf_free(sc, re->m_fifo[re->m_fifo_tail]);
re->m_fifo_depth--;
/* XXX check it's not < 0 */
re->m_fifo[re->m_fifo_tail] = NULL;
}
bf = ath_edma_rxbuf_alloc(sc);
/* XXX should ensure the FIFO is not NULL? */
if (bf == NULL) {
ath: Minor style cleanups device_printf => DPRINTF and two whitespace adjustments Submitted by: Augustin Cavalier <waddlesplash@gmail.com> Obtained from: Haiku (4a88aa503ad4155a20931e263d24343043994ea9) MFC after: 1 week
2018-08-10 13:38:23 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX,
Log some more information when the RX buffer allocation failed.
2013-03-18 01:11:52 +00:00
"%s: Q%d: alloc failed: i=%d, nbufs=%d?\n",
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
__func__,
Log some more information when the RX buffer allocation failed.
2013-03-18 01:11:52 +00:00
qtype,
i,
nbufs);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
break;
}
re->m_fifo[re->m_fifo_tail] = bf;
/* Write to the RX FIFO */
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
DPRINTF(sc, ATH_DEBUG_EDMA_RX,
"%s: Q%d: putrxbuf=%p (0x%jx)\n",
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
__func__,
qtype,
Fix the busdma logic to work with EDMA chipsets when using bounce buffers (ie, >4GB on amd64.) The underlying problem was that PREREAD doesn't sync the mbuf with the DMA memory (ie, bounce buffer), so the bounce buffer may have had stale information. Thus it was always considering the buffer completed and things just went off the rails. This change does the following: * Make ath_rx_pkt() always consume the mbuf somehow; it no longer passes error mbufs (eg CRC errors, crypt errors, etc) back up to the RX path to recycle. This means that a new mbuf is always allocated each time, but it's cleaner. * Push the RX buffer map/unmap to occur in the RX path, not ath_rx_pkt(). Thus, ath_rx_pkt() now assumes (a) it has to consume the mbuf somehow, and (b) that it's already been unmapped and synced. * For the legacy path, the descriptor isn't mapped, it comes out of coherent, DMA memory anyway. So leave it there. * For the EDMA path, the RX descriptor has to be cleared before its passed to the hardware, so that when we check with a POSTREAD sync, we actually get either a blank (not finished) or a filled out descriptor (finished.) Otherwise we get stale data in the DMA memory. * .. so, for EDMA RX path, we need PREREAD|PREWRITE to sync the data -> DMA memory, then POSTREAD|POSTWRITE to finish syncing the DMA memory -> data. * Whilst we're here, make sure that in EDMA buffer setup (ie, bzero'ing the descriptor part) is done before the mbuf is map/synched. NOTE: there's been a lot of commits besides this one with regards to tidying up the busdma handling in ath(4). Please check the recent commit history. Discussed with and thanks to: scottl Tested: * AR5416 (non-EDMA) on i386, with the DMA tag for the driver set to 2^^30, not 2^^32, STA * AR9580 (EDMA) on i386, as above, STA * User - tested AR9380 on amd64 with 32GB RAM. PR: kern/177530
2013-04-04 08:21:56 +00:00
bf->bf_desc,
(uintmax_t) bf->bf_daddr);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
ath_hal_putrxbuf(sc->sc_ah, bf->bf_daddr, qtype);
re->m_fifo_depth++;
INCR(re->m_fifo_tail, re->m_fifolen);
}
/*
* Return how many were allocated.
*/
DPRINTF(sc, ATH_DEBUG_EDMA_RX, "%s: Q%d: nbufs=%d, nalloced=%d\n",
__func__,
qtype,
nbufs,
i);
return (i);
}
static int
ath_edma_rxfifo_flush(struct ath_softc *sc, HAL_RX_QUEUE qtype)
{
struct ath_rx_edma *re = &sc->sc_rxedma[qtype];
int i;
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK_ASSERT(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
for (i = 0; i < re->m_fifolen; i++) {
if (re->m_fifo[i] != NULL) {
#ifdef ATH_DEBUG
Fix build when ATH_DEBUG is not defined.
2012-07-10 18:57:05 +00:00
struct ath_buf *bf = re->m_fifo[i];
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
if (sc->sc_debug & ATH_DEBUG_RECV_DESC)
ath_printrxbuf(sc, bf, 0, HAL_OK);
#endif
ath_edma_rxbuf_free(sc, re->m_fifo[i]);
re->m_fifo[i] = NULL;
re->m_fifo_depth--;
}
}
if (re->m_rxpending != NULL) {
m_freem(re->m_rxpending);
re->m_rxpending = NULL;
}
re->m_fifo_head = re->m_fifo_tail = re->m_fifo_depth = 0;
return (0);
}
/*
* Setup the initial RX FIFO structure.
*/
static int
ath_edma_setup_rxfifo(struct ath_softc *sc, HAL_RX_QUEUE qtype)
{
struct ath_rx_edma *re = &sc->sc_rxedma[qtype];
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK_ASSERT(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
if (! ath_hal_getrxfifodepth(sc->sc_ah, qtype, &re->m_fifolen)) {
device_printf(sc->sc_dev, "%s: qtype=%d, failed\n",
__func__,
qtype);
return (-EINVAL);
}
Quieten the RX/TX descriptor and FIFO setup debugging. Tested: * AR9485, STA mode
2014-05-05 08:00:50 +00:00
if (bootverbose)
device_printf(sc->sc_dev,
"%s: type=%d, FIFO depth = %d entries\n",
__func__,
qtype,
re->m_fifolen);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Allocate ath_buf FIFO array, pre-zero'ed */
Revert r327828, r327949, r327953, r328016-r328026, r328041: Uses of mallocarray(9). The use of mallocarray(9) has rocketed the required swap to build FreeBSD. This is likely caused by the allocation size attributes which put extra pressure on the compiler. Given that most of these checks are superfluous we have to choose better where to use mallocarray(9). We still have more uses of mallocarray(9) but hopefully this is enough to bring swap usage to a reasonable level. Reported by: wosch PR: 225197
2018-01-21 15:42:36 +00:00
re->m_fifo = malloc(sizeof(struct ath_buf *) * re->m_fifolen,
M_ATHDEV,
M_NOWAIT | M_ZERO);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
if (re->m_fifo == NULL) {
device_printf(sc->sc_dev, "%s: malloc failed\n",
__func__);
return (-ENOMEM);
}
/*
* Set initial "empty" state.
*/
re->m_rxpending = NULL;
re->m_fifo_head = re->m_fifo_tail = re->m_fifo_depth = 0;
return (0);
}
static int
ath_edma_rxfifo_free(struct ath_softc *sc, HAL_RX_QUEUE qtype)
{
struct ath_rx_edma *re = &sc->sc_rxedma[qtype];
device_printf(sc->sc_dev, "%s: called; qtype=%d\n",
__func__,
qtype);
ath: Minor style cleanups device_printf => DPRINTF and two whitespace adjustments Submitted by: Augustin Cavalier <waddlesplash@gmail.com> Obtained from: Haiku (4a88aa503ad4155a20931e263d24343043994ea9) MFC after: 1 week
2018-08-10 13:38:23 +00:00
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
free(re->m_fifo, M_ATHDEV);
return (0);
}
static int
ath_edma_dma_rxsetup(struct ath_softc *sc)
{
int error;
Fix EDMA RX to actually work without panicing the machine. I was setting up the RX EDMA buffer to be 4096 bytes rather than the RX data buffer portion. The hardware was likely getting very confused and DMAing descriptor portions into places it shouldn't, leading to memory corruption and occasional panics. Whilst here, don't bother allocating descriptors for the RX EDMA case. We don't use those descriptors. Instead, just allocate ath_buf entries.
2012-07-14 02:07:51 +00:00
/*
* Create RX DMA tag and buffers.
*/
error = ath_descdma_setup_rx_edma(sc, &sc->sc_rxdma, &sc->sc_rxbuf,
"rx", ath_rxbuf, sc->sc_rx_statuslen);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
if (error != 0)
return error;
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
(void) ath_edma_setup_rxfifo(sc, HAL_RX_QUEUE_HP);
(void) ath_edma_setup_rxfifo(sc, HAL_RX_QUEUE_LP);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_UNLOCK(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
return (0);
}
static int
ath_edma_dma_rxteardown(struct ath_softc *sc)
{
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_LOCK(sc);
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
ath_edma_flush_deferred_queue(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
ath_edma_rxfifo_flush(sc, HAL_RX_QUEUE_HP);
ath_edma_rxfifo_free(sc, HAL_RX_QUEUE_HP);
ath_edma_rxfifo_flush(sc, HAL_RX_QUEUE_LP);
ath_edma_rxfifo_free(sc, HAL_RX_QUEUE_LP);
Change the RX EDMA path to first complete the FIFO, then re-populate it with fresh descriptors, before handling the frames. Wrap it all in the RX locks. Since the FIFO is very shallow (16 for HP, 128 for LP) it needs to be drained and replenished very quickly. Ideally, I'll eventually move this RX FIFO drain/fill into the interrupt handler, only deferring the actual frame completion.
2012-07-14 02:52:48 +00:00
ATH_RX_UNLOCK(sc);
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Free RX ath_buf */
/* Free RX DMA tag */
if (sc->sc_rxdma.dd_desc_len != 0)
ath_descdma_cleanup(sc, &sc->sc_rxdma, &sc->sc_rxbuf);
return (0);
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
}
void
ath_recv_setup_edma(struct ath_softc *sc)
{
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
/* Set buffer size to 4k */
sc->sc_edma_bufsize = 4096;
/* Fetch EDMA field and buffer sizes */
(void) ath_hal_getrxstatuslen(sc->sc_ah, &sc->sc_rx_statuslen);
Fix EDMA RX to actually work without panicing the machine. I was setting up the RX EDMA buffer to be 4096 bytes rather than the RX data buffer portion. The hardware was likely getting very confused and DMAing descriptor portions into places it shouldn't, leading to memory corruption and occasional panics. Whilst here, don't bother allocating descriptors for the RX EDMA case. We don't use those descriptors. Instead, just allocate ath_buf entries.
2012-07-14 02:07:51 +00:00
/* Configure the hardware with the RX buffer size */
(void) ath_hal_setrxbufsize(sc->sc_ah, sc->sc_edma_bufsize -
sc->sc_rx_statuslen);
Quieten the RX/TX descriptor and FIFO setup debugging. Tested: * AR9485, STA mode
2014-05-05 08:00:50 +00:00
if (bootverbose) {
device_printf(sc->sc_dev, "RX status length: %d\n",
sc->sc_rx_statuslen);
device_printf(sc->sc_dev, "RX buffer size: %d\n",
sc->sc_edma_bufsize);
}
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
sc->sc_rx.recv_stop = ath_edma_stoprecv;
sc->sc_rx.recv_start = ath_edma_startrecv;
sc->sc_rx.recv_flush = ath_edma_recv_flush;
sc->sc_rx.recv_tasklet = ath_edma_recv_tasklet;
sc->sc_rx.recv_rxbuf_init = ath_edma_rxbuf_init;
Implement EDMA RX for AR93xx and later chips. This is inspired by ath9k and the reference driver, but it's a new implementation of the RX FIFO handling. This has some issues - notably the FIFO needs to be reprogrammed when the chip is reset.
2012-07-10 00:08:39 +00:00
sc->sc_rx.recv_setup = ath_edma_dma_rxsetup;
sc->sc_rx.recv_teardown = ath_edma_dma_rxteardown;
Break out the RX completion path into "FIFO check / refill" and "complete RX frames." The 128 entry RX FIFO is really easy to fill up and miss refilling when it's done in the ath taskq - as that gets blocked up doing RX completion, TX completion and other random things. So the 128 entry RX FIFO now gets emptied and refilled in the ath_intr() task (and it grabs / releases locks, so now ath_intr() can't just be a FAST handler yet!) but the locks aren't held for very long. The completion part is done in the ath taskqueue context. Details: * Create a new completed frame list - sc->sc_rx_rxlist; * Split the EDMA RX process queue into two halves - one that processes the RX FIFO and refills it with new frames; another that completes the completed frame list; * When tearing down the driver, flush whatever is in the deferred queue as well as what's in the FIFO; * Create two new RX methods - one that processes all RX queues, one that processes the given RX queue. When MSI is implemented, we get told which RX queue the interrupt came in on so we can specifically schedule that. (And I can do that with the non-MSI path too; I'll figure that out later.) * Convert the legacy code over to use these new RX methods; * Replace all the instances of the RX taskqueue enqueue with a call to a relevant RX method to enqueue one or all RX queues. Tested: * AR9380, STA * AR9580, STA * AR5413, STA
2013-03-19 19:32:28 +00:00
sc->sc_rx.recv_sched = ath_edma_recv_sched;
sc->sc_rx.recv_sched_queue = ath_edma_recv_sched_queue;
Begin abstracting out the RX path in preparation for RX EDMA support. The RX EDMA support requires a modified approach to the RX descriptor handling. Specifically: * There's now two RX queues - high and low priority; * The RX queues are implemented as FIFOs; they're now an array of pointers to buffers; * .. and the RX buffer and descriptor are in the same "buffer", rather than being separate. So to that end, this commit abstracts out most of the RX related functions from the bulk of the driver. Notably, the RX DMA/buffer allocation isn't updated, primarily because I haven't yet fleshed out what it should look like. Whilst I'm here, create a set of matching but mostly unimplemented EDMA stubs. Tested: * AR9280, station mode TODO: * Thorough AP and other mode testing for non-EDMA chips; * Figure out how to allocate RX buffers suitable for RX EDMA, including correctly setting the mbuf length to compensate for the RX descriptor and completion status area.
2012-07-03 06:59:12 +00:00
}
Reference in a new issue Copy permalink