On fault the driver is given the opportunity to perform any operation
it sees fit in order to place the buffer into a CPU visible area of
memory. This patch doesn't break TTM users, nouveau, vmwgfx and radeon
should keep working properly. Future patch will take advantage of this
infrastructure and remove the old path from TTM once driver are
converted.
V2 return VM_FAULT_NOPAGE if callback return -EBUSY or -ERESTARTSYS
V3 balance io_mem_reserve and io_mem_free call, fault_reserve_notify
is responsible to perform any necessary task for mapping to succeed
V4 minor cleanup, atomic_t -> bool as member is protected by reserve
mecanism from concurent access
V5 the callback is now responsible for iomapping the bo and providing
a virtual address this simplify TTM and will allow to get rid of
TTM_MEMTYPE_FLAG_NEEDS_IOREMAP
V6 use the bus addr data to decide to ioremap or this isn't needed
but we don't necesarily need to ioremap in the callback but still
allow driver to use static mapping
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
There is case where we want to be able to wait only for the
GPU while not waiting for other buffer to be unreserved. This
patch split the no_wait argument all the way down in the whole
ttm path so that upper level can decide on what to wait on or
not.
[airlied: squashed these 4 for bisectability reasons.]
drm/radeon/kms: update to TTM no_wait splitted argument
drm/nouveau: update to TTM no_wait splitted argument
drm/vmwgfx: update to TTM no_wait splitted argument
[vmwgfx patch: Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>]
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Replace sequential calls to kobject_init() and kobject_add() with the
combo wrapper kobject_init_and_add(), which provides the same
semantics.
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Now that the drm core can do this, lets just use it, split the code out
so TTM doesn't have to drag all of drmP.h in.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Constify struct sysfs_ops.
This is part of the ops structure constification
effort started by Arjan van de Ven et al.
Benefits of this constification:
* prevents modification of data that is shared
(referenced) by many other structure instances
at runtime
* detects/prevents accidental (but not intentional)
modification attempts on archs that enforce
read-only kernel data at runtime
* potentially better optimized code as the compiler
can assume that the const data cannot be changed
* the compiler/linker move const data into .rodata
and therefore exclude them from false sharing
Signed-off-by: Emese Revfy <re.emese@gmail.com>
Acked-by: David Teigland <teigland@redhat.com>
Acked-by: Matt Domsch <Matt_Domsch@dell.com>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Acked-by: Hans J. Koch <hjk@linutronix.de>
Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>
Acked-by: Jens Axboe <jens.axboe@oracle.com>
Acked-by: Stephen Hemminger <shemminger@vyatta.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
- Without this change I get a general protection fault.
- Also use PTR_ERR where applicable.
Signed-off-by: Maarten Maathuis <madman2003@gmail.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Cc: stable@kernel.org
Signed-off-by: Dave Airlie <airlied@redhat.com>
http://bugzilla.kernel.org/show_bug.cgi?id=15328
This fixes a serious regression on AGP/non-PAT systems, where
pages were ending up in the wrong state and slowing down the
whole system.
[airlied: taken this from the bug as the other option is to revert
the change which caused it].
Tested-by: John W. Linville (in bug).
Signed-off-by: Dave Airlie <airlied@redhat.com>
If the buffer object was already in the requested memory type, but
outside of the requested range it was never moved into the requested range.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6:
drm/radeon/kms: Fix oops after radeon_cs_parser_init() failure.
drm/radeon/kms: move radeon KMS on/off switch out of staging.
drm/radeon/kms: Bailout of blit if error happen & protect with mutex V3
drm/vmwgfx: Don't send bad flags to the host
drm/vmwgfx: Request SVGA version 2 and bail if not found
drm/vmwgfx: Correctly detect 3D
drm/ttm: remove unnecessary save_flags and ttm_flag_masked in ttm_bo_util.c
drm/kms: Remove incorrect comment in struct drm_mode_modeinfo
drm/ttm: remove padding from ttm_ref_object on 64bit builds
drm/radeon/kms: release agp on error.
drm/kms/radeon/agp: Move the check of the aper_size after drm_acp_acquire and drm_agp_info
drm/kms/radeon/agp: Fix warning, format ‘%d’ expects type ‘int’, but argument 4 has type ‘size_t’
drm/ttm: Avoid conflicting reserve_memtype during ttm_tt_set_page_caching.
drm/kms/radeon: pick digitial encoders smarter. (v3)
drm/radeon/kms: use active device to pick connector for encoder
drm/radeon/kms: fix incorrect logic in DP vs eDP connector checking.
Re-order structure ttm_ref_object to remove 8 bytes of alignment padding
on 64 bit builds, so shrinking its size from 72 to 64 bytes allowing it
to fit into a smaller slab.
Signed-off-by: Richard Kennedy <richard@rsk.demon.co.uk>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Fixes errors like:
> reserve_ram_pages_type failed 0x15b7a000-0x15b7b000, track 0x8, req 0x10
when a BO is moved between WC and UC areas.
Reported-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (95 commits)
drm/radeon/kms: preface warning printk with driver name
drm/radeon/kms: drop unnecessary printks.
drm: fix regression in fb blank handling
drm/radeon/kms: make hibernate work on IGPs
drm/vmwgfx: Optimize memory footprint for DMA buffers.
drm/ttm: Allow system memory as a busy placement.
drm/ttm: Fix race condition in ttm_bo_delayed_delete (v3, final)
drm/nv50: prevent switching off SOR when in use for DVI-over-DP
drm/nv50: fail auxch transaction if reply count not what we expect
drm/nouveau: fix failure path if userspace specifies no valid memtypes
drm/nouveau: report LVDS as disconnected if lid closed
drm/radeon/kms: fix legacy get_engine/memory clock
drm/radeon/kms/atom: atom parser fixes
drm/radeon/kms: clean up atombios pll code
drm/radeon/kms: clean up pll struct
drm/radeon/kms/atom: fix crtc lock ordering
drm/radeon: r6xx/r7xx possible security issue, system ram access
drm/radeon/kms: r600/r700 don't test ib if ib initialization fails
drm/radeon/kms: Forbid creation of framebuffer with no valid GEM object
drm/radeon/kms: r600 handle irq vector ring overflow
...
This is needed to fix a vmwgfx memory usage bug.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Resending this with Thomas Hellstrom's signoff for merging into 2.6.33
ttm_bo_delayed_delete has a race condition, because after we do:
kref_put(&nentry->list_kref, ttm_bo_release_list);
we are not holding the list lock and not holding any reference to
objects, and thus every bo in the list can be removed and freed at
this point.
However, we then use the next pointer we stored, which is not guaranteed
to be valid.
This was apparently the cause of some Nouveau oopses I experienced.
This patch rewrites the function so that it keeps the reference to nentry
until nentry itself is freed and we already got a reference to nentry->next.
v2 updated by me according to Thomas Hellstrom's feedback.
v3 proposed by Thomas Hellstrom. Commit comment updated by me.
Both updates fixed minor efficiency/style issues only and all three versions
should be correct.
Signed-off-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This is a convention that the vmwgfx driver has come to rely on.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This is needed for a bugfix in the vmwgfx driver.
Drivers may have GPU bindings on buffers that core TTM is not aware of,
and TTM may view those buffers as ordinary system memory buffers.
Add a notifier to such drivers when TTM is about to move the buffer
contents out to swappable memory. The driver must then release any
private GPU bindings on those buffers.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6:
drm/radeon/kms: fix r100->r500 CS checker for compressed textures. (v2)
drm/radeon/kms: allow for texture tiling
drm/radeon/kms: init pm on all chipsets
drm/radeon/kms: HDMI support for R600 KMS
drm/radeon/kms: make sure mc is initialized before mapping blit bo
drm/radeon/kms: Return to userspace on ERESTARTSYS
drm/radeon/gem: don't leak a gem object if reserve fails on get tiling (v2)
drm/radeon/kms: don't report allocate failure on ERESTARTSYS
drm/radeon/kms: Check if bo we got from ttm are radeon object or not
drm/radeon/kms: If no placement is supplied fallback to system
drm/ttm: Fix memory type manager debug information printing
drm/ttm: Fix printk format & compute bo->mem.size at bo initialization
drm/ttm: Fix potential ttm_mem_evict_first races.
drm/ttm: Delayed delete fixes.
drm/ttm: fix two bugs in new placement routines.
drm/ttm: fix incorrect logic in ttm_bo_io path
drm/nouveau: remove use of -ERESTART
nouveau: Fix endianness with new context program loader
drm/nouveau: fix build with CONFIG_AGP=n
drm/nouveau: fix ch7006 build
System memory type doesn't have a drm_mm manager associated to
it. This patch avoid trying to call drm_mm_debug on unitialized
drm_mm when printing debug info on the system memory manager.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
1) The function was previously called with a potentially empty
LRU list which would have lead to an OOPS or servere corruption.
2) In rare cases, after reservation has succeeded, another process may
already have evicted it or even pinned it. We must revalidate the
buffer status after releasing the lru lock.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
1) Remove from lru before reserving so we avoid competing with
evicting processes.
2) Avoid calling kref_put() on bo::list_kref while spinlocked.
3) Additional refcounting bug-checking.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
a) the loops were going to <= not <, leading to illegal memory access
b) the busy placement checks were using the placement arrays not the
busy placement ones.
Acked-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This path isn't used by radeon yet, but future drivers will want it,
so fix it right.
Reported-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6:
drm/ttm: export some functions useful to drivers using ttm
drm/radeon/kms/avivo: fix typo in new_pll module description
drm/radeon/kms: Convert radeon to new ttm_bo_init
drm/ttm: Convert ttm_buffer_object_init to use ttm_placement
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (189 commits)
drm/radeon/kms: fix warning about cur_placement being uninitialised.
drm/ttm: Print debug information on memory manager when eviction fails
drm: Add memory manager debug function
drm/radeon/kms: restore surface registers on resume.
drm/radeon/kms/r600/r700: fallback gracefully on ucode failure
drm/ttm: Initialize eviction placement in case the driver callback doesn't
drm/radeon/kms: cleanup structure and module if initialization fails
drm/radeon/kms: actualy set the eviction placements we choose
drm/radeon/kms: Fix NULL ptr dereference
drm/radeon/kms/avivo: add support for new pll selection algo
drm/radeon/kms/avivo: fix some bugs in the display bandwidth setup
drm/radeon/kms: fix return value from fence function.
drm/radeon: Remove tests for -ERESTART from the TTM code.
drm/ttm: Have the TTM code return -ERESTARTSYS instead of -ERESTART.
drm/radeon/kms: Convert radeon to new TTM validation API (V2)
drm/ttm: Rework validation & memory space allocation (V3)
drm: Add search/get functions to get a block in a specific range
drm/radeon/kms: fix avivo tiling regression since radeon object rework
drm/i915: Remove a debugging printk from hangcheck
drm/radeon/kms: make sure i2c id matches
...
These are functions required by nouveau which will be merged later.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Convert ttm_buffer_object_init to use struct ttm_placement and
rename to ttm_bo_init for consistency with function naming. This
allow to give more complex placement at buffer creation. For
instance you ask to allocate bo into vram first but if there is
not enough vram you can give system as a second possible
placement. It also allow to create buffer in a specific range.
Also rename ttm_buffer_object_validate to ttm_bo_validate.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This add helper function to print information on eviction placements
and memory manager status when eviction fails to allocate memory
space.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This would allow to catch driver callback error of not properly
setting the eviction placement structure.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@linux.ie>
Return -ERESTARTSYS instead of -ERESTART when interrupted by a signal.
The -ERESTARTSYS is converted to an -EINTR by the kernel signal layer
before returned to user-space.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This change allow driver to pass sorted memory placement,
from most prefered placement to least prefered placement.
In order to avoid long function prototype a structure is
used to gather memory placement informations such as range
restriction (if you need a buffer to be in given range).
Range restriction is determined by fpfn & lpfn which are
the first page and last page number btw which allocation
can happen. If those fields are set to 0 ttm will assume
buffer can be put anywhere in the address space (thus it
avoids putting a burden on the driver to always properly
set those fields).
This patch also factor few functions like evicting first
entry of lru list or getting a memory space. This avoid
code duplication.
V2: Change API to use placement flags and array instead
of packing placement order into a quadword.
V3: Make sure we set the appropriate mem.placement flag
when validating or allocation memory space.
[Pending Thomas Hellstrom further review but okay
from preliminary review so far].
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
None of the in-tree drivers use user objects yet so this wasn't hitting
us.
Stanse found unreachable code in ttm_bo_add_ttm:
http://decibel.fi.muni.cz/~xslaby/stanse/error.cgi?db=32&id=714#l238
Reported-by: Jiri Slaby <jirislaby@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Utilities to reserve, unreserve and fence a list of TTM
buffer objects in a deadlock-safe manner.
Used by the vmwgfx driver.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This is intended to be used by ttm-aware drivers to
1) Block clients to inactive masters when
they try to validate buffers for GPU use.
2) Optionally block clients to the current master when
there is thrashing due to GPU memory shortage.
Used by the vmwgfx driver.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Add objects needed for user-space to maintain reference counts on ttm objects.
This is used by the vmwgfx driver which allows user-space to maintain
map-counts on dma buffers, lock-counts on the ttm lock and ref-counts on
gpu surfaces, gpu contexts and dma buffer.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
I moved the allocation until after the check for (si->totalhigh == 0).
Signed-off-by: Dan Carpenter <error27@gmail.com>
Acked-By: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (52 commits)
drm/kms: Init the CRTC info fields for modes forced from the command line.
drm/radeon/r600: CS parser updates
drm/radeon/kms: add debugfs for power management for AtomBIOS devices
drm/radeon/kms: initial mode validation support
drm/radeon/kms/atom/dce3: call transmitter init on mode set
drm/radeon/kms: store detailed connector info
drm/radeon/kms/atom/dce3: fix up usPixelClock calculation for Transmitter tables
drm/radeon/kms/r600: fix rs880 support v2
drm/radeon/kms/r700: fix some typos in chip init
drm/radeon/kms: remove some misleading debugging output
drm/radeon/kms: stop putting VRAM at 0 in MC space on r600s.
drm/radeon/kms: disable D1VGA and D2VGA if enabled
drm/radeon/kms: Don't RMW CP_RB_CNTL
drm/radeon/kms: fix coherency issues on AGP cards.
drm/radeon/kms: fix rc410 suspend/resume.
drm/radeon/kms: add quirk for hp dc5750
drm/radeon/kms/atom: fix potential oops in spread spectrum code
drm/kms: typo fix
drm/radeon/kms/atom: Make card_info per device
drm/radeon/kms/atom: Fix DVO support
...
When we are evicting from VRAM->RAM we allocate the ttm object,
but we don't set the caching policy on it before blitting into it.
This means on AGP we end up blitting into cached pages, and
the CPU later flushes out on top of them. This was mostly seen as
font corruption.
The other question is why we don't evict VRAM->GTT in a lot of cases,
this would save us some cache transitions since a lot of objects
that are evicted from VRAM will probably end up being pulled back in
a few operations later, and evicting them to system memory involves
2 unnecessary cache transitions.
Signed-off-by: Dave Airlie <airlied@redhat.com>
conflict in radeon since new init path merged with vga arb code.
Conflicts:
drivers/gpu/drm/radeon/radeon.h
drivers/gpu/drm/radeon/radeon_asic.h
drivers/gpu/drm/radeon/radeon_device.c
the global refcount wasn't being increased after the first reference.
this caused an oops on unload on a multi-gpu card.
Signed-off-by: Dave Airlie <airlied@redhat.com>
* mark struct vm_area_struct::vm_ops as const
* mark vm_ops in AGP code
But leave TTM code alone, something is fishy there with global vm_ops
being used.
Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
