Commit 6960e6da authored by Gerd Hoffmann's avatar Gerd Hoffmann
Browse files

drm: fix drm_mode_addfb() on big endian machines. 



Userspace on big endian machhines typically expects the ADDFB ioctl
returns a big endian framebuffer.  drm_mode_addfb() will call
drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
values though, which is wrong.  This patch fixes that.

Drivers (both kernel and xorg) have quirks in place to deal with the
broken drm_mode_addfb() behavior.  Because of this we can't just change
drm_mode_addfb() behavior for everybody without breaking things.  Add
the quirk_addfb_prefer_host_byte_order field to mode_config, so drivers
can opt-in.

Signed-off-by: default avatarGerd Hoffmann <kraxel@redhat.com>
Reviewed-by: default avatarDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20180905060445.15008-5-kraxel@redhat.com
parent 00409fd6

drivers/gpu/drm/drm_framebuffer.c

+11 −0
Original line number Diff line number Diff line
@@ -124,6 +124,17 @@ int drm_mode_addfb(struct drm_device *dev, struct drm_mode_fb_cmd *or, 
	    r.pixel_format == DRM_FORMAT_XRGB2101010)

		r.pixel_format = DRM_FORMAT_XBGR2101010;



	if (dev->mode_config.quirk_addfb_prefer_host_byte_order) {

		if (r.pixel_format == DRM_FORMAT_XRGB8888)

			r.pixel_format = DRM_FORMAT_HOST_XRGB8888;

		if (r.pixel_format == DRM_FORMAT_ARGB8888)

			r.pixel_format = DRM_FORMAT_HOST_ARGB8888;

		if (r.pixel_format == DRM_FORMAT_RGB565)

			r.pixel_format = DRM_FORMAT_HOST_RGB565;

		if (r.pixel_format == DRM_FORMAT_XRGB1555)

			r.pixel_format = DRM_FORMAT_HOST_XRGB1555;

	}



	ret = drm_mode_addfb2(dev, &r, file_priv);

	if (ret)

		return ret;

include/drm/drm_mode_config.h

+14 −0
Original line number Diff line number Diff line
@@ -811,6 +811,20 @@ struct drm_mode_config { 
	uint32_t preferred_depth, prefer_shadow;

	bool quirk_addfb_prefer_xbgr_30bpp;



	/**

	 * @quirk_addfb_prefer_host_byte_order:

	 *

	 * When set to true drm_mode_addfb() will pick host byte order

	 * pixel_format when calling drm_mode_addfb2().  This is how

	 * drm_mode_addfb() should have worked from day one.  It

	 * didn't though, so we ended up with quirks in both kernel

	 * and userspace drivers to deal with the broken behavior.

	 * Simply fixing drm_mode_addfb() unconditionally would break

	 * these drivers, so add a quirk bit here to allow drivers

	 * opt-in.

	 */

	bool quirk_addfb_prefer_host_byte_order;



	/**

	 * @async_page_flip: Does this device support async flips on the primary

	 * plane?

CRA Git | Maintained and supported by SUSTech CRA and CCSE