modesetting: Only use modifiers on kms drivers which do support them

Graphics / X.Org / X.Org Server - Mario Kleiner [] - 23 April 2018 18:05 EDT

Use the DRM_CAP_ADDFB2_MODIFIERS query to make sure the kms driver supports modifiers in the addfb2 ioctl, and fall back to addfb ioctl without modifiers if modifiers are unsupported.

E.g., as of Linux 4.17, nouveau-kms so far does not suppport modifiers and gets angry if drmModeAddFB2WithModifiers() is called (-> failure to set a video mode -> blank screen), but Mesa's nvc0+ gallium driver causes gbm_bo_get_modifier() to return a valid modifier by translating the default tiling of bo's created via gbm_bo_create() into a modifier other than DRM_FORMAT_MOD_INVALID (see Mesa's nvc0_miptree_get_modifier()).

Testing for != DRM_FORMAT_MOD_INVALID is apparently not sufficient for safe use of drmModeAddFB2WithModifiers.

Bonus: Handle potential failure of populate_format_modifiers().

The required DRM_CAP is defined since libdrm v2.4.65, and we require v2.4.89+ for the server, so we can use it unconditionally.

Tested on intel-kms, radeon-kms, nouveau-kms. Fixes failure on NVidia Pascal.

Fixes: 2f807c2324b4 ("modesetting: Add support for multi-plane pixmaps when page-flipping")

e29d78327 modesetting: Only use modifiers on kms drivers which do support them.
hw/xfree86/drivers/modesetting/driver.c | 5 +++++
hw/xfree86/drivers/modesetting/driver.h | 2 ++
hw/xfree86/drivers/modesetting/drmmode_display.c | 13 ++++++-------
3 files changed, 13 insertions(+), 7 deletions(-)


  • Share