Graphics

SprezzOS contains powerful graphics support.

Raster displays (primarily LEDs and CRTs) underpin most users' interactions with their desktops and mobile devices. Modern desktops regularly drive one or more large, colorful, high-resolution LEDs via one or sometimes multiple video cards. Graphics processing units commonly contain more transistors than the CPUs which control them. Despite this, the basic process interface exposed by ANSI C and the UNIX kernel consists of two character streams.

Don't blame me, man; I didn't do it.

• FIXME Explain the VESA BIOS Extensions!
• FIXME Explain how printing integrates with all this...

Displays

Displays can typically identify themselves to a compliant EDID (Extended Display Identification Data) parser, usually using some instance of the DDC (Data Display Channel) protocol suite. DDC was superseded by E-DDC in 1999, but is still usually referred to as DDC. EDID itself was split out from DDC with the release of DDC2 in April 1996. EDID 1.4 was superseded by E-EDID (EDID 2.0), but this is similarly referred to as simply EDID. DisplayID is a new definition designed to replace E-EDID, but implementation has been slow.

On DE15 (commonly known as VGA) connectors, DDC takes over the 4 "identification" pins from the IBM VGA definition (ID0--ID3, pins 11, 12, 4 and 15). These pins were not commonly used under their original semantics. The similarly rare DDC1 implements a serial link atop these pins, continuously retransmitting the EDID block on Pin 12, clocked against vertical sync. DDC2 implements an I²C bus, using Pin 12 for data and Pin 15 as a clock.

DDC2B is also implemented by DVI, HDMI, and DisplayPort.

Marketed Resolutions

Note that these terms are largely meaningless, and have been since SuperVGA (also, where's EGA?).

Physical Connectors

Name	Picture	Standards body	Signal	Data
DE15 (High-density DE-shell, 15 pins)		N/A	Analog	DDC (serial link) / DDC2B (I²C link)
Flat Panel Display Link (FPD-Link) over LVDS (Low Voltage Differential Signaling) (also the defunct LDI and OpenLDI)	N/A	NatSemi / Silicon Valley Analog (FPD-Link) ANSI/TIA/EIA-644-A (LVDS)	Digital	DDC/CI
DVI (Digital Visual Interface)		Digital Display Working Group (royalty-free)	Digital (DVI-D) / Analog (DVI-A) / Digital + Analog (DVI-I)	DDC2B
HDMI (High-Definiton Multimedia Interface)		HDMI Forum (royalty + registration)	Digital	E-DDC + HDCP DDC
DisplayPort		VESA (royalty-free)	Digital	DDC

Color Management

FIXME!

Character Cell Output

A rather lovely Wyse 50 serial terminal. An elegant weapon, for a more civilized age.

Glyph matching

Libraries such as libaa and libcaca make use of glyph matching algorithms to provide a generic character cell-based rasterizer (libcaca includes Unicode support and colors, among other advances). The results can be quite striking!

Raster Output

Successful configuration of the Linux framebuffer at boot will display a mascot for each CPU.

Graphic output will require some kernel module(s) driving the video device(s). Framebuffer drivers are independent of userspace programs. They create a /dev entry, can support the system console, and are targeted by SDL, DirectFB, and fbdev (the framebuffer X server). Direct Rendering Manager (DRM) drivers are used as part of X11's Direct Rendering Infrastructure (DRI), and require their own X servers. Framebuffer drivers generally accelerate only 2D operations (blitting, rectangular tiling, etc), or none at all. DRM drivers expose a much richer API, and can be used with fully hardware-accelerated X servers. In addition, closed source drivers -- usually paired with their own X servers, and not typically supporting a framebuffer device -- can be used. Use of both the DRI and framebuffer drivers with a device can lead to errors, although sometimes it works fine.

Framebuffer

An abstraction of hardware, mapping memory to displayed pixels. If a framebuffer driver is active, /dev/fbX devices will exist, and the fbset tool can be used to display their statuses. It is generally not advised to use a framebuffer driver at the same time as X-based hardware control, though this sometimes works in practice. When a framebuffer is present and supported at kernel boot time, Linux will display a Tux logo (see right) for each processor in the machine. The framebuffer mode is generally configured via the kernel command line or arguments to modprobe(1)/init_module(2).

X.org Server 1.13 was released 2012-09-05.

DRI is a combination of DRM, X.org, and Mesa.

X architecture as of 2012

Xorg

Xorg's code base combines the DDX (Device-Dependent X) and the DIX (Device-Independent X). It is a network-transparent client-server reference implementation of version 11 of the X Protocol, and provides a complete display and interface system capable of supporting multiple displays and multiple input devices. Multiple instances of the X Server can be run on one machine without much difficulty, though they must control different hardware resources. XNest and Xephyr are X servers targeting X windows as their framebuffers. Xvfb provides a null X server, allowing headless execution of Xorg-based programs.

• Extensions
  • Damage -- Notification that a portion of a window needs to be redrawn.
  • Render -- Porter-Duff image compositioning. It wraps the Pixmap and Window datatypes with a new type, Picture. Picture provides an alpha channel. This is necessary for any kind of translucency within an X server, a prerequisite of (among other things) sane anti-aliased text support. For that matter, it added anti-aliased drawing primitives.
  • Shape -- Support for non-rectangular windows.
  • RandR -- The X Resize, Rotate and Reflect Extension allows manipulation of root screens. Used for hotplugging devices, multidisplay, and changing devices.
  • Xv (X Video) -- Scaling and YUV->RGB conversion offloading.
  • XvMC -- X Video Motion Compensation. Offloads mocomp and iDCT for MPEG2.
  • GLX -- The OpenGL extension, providing support for use of OpenGL within an X window. It is superseded by EGL.
    • The Microsoft Windows analogue of GLX is WGL
    • The Mac OS X analogue of GLX is CGL ("Core OpenGL").
    • The original SGI implementation was IRIS GL ("Integrated Raster Imaging Systems GL").
  • XDMCP -- A good way to get rooted. Don't use XDMCP. You will be laughed at, and beaten, and made to regret having done so.
  • Xft -- The X Font Extension. It drove Render, but has been largely replaced by cairo.
  • Composite -- Allows a compositing manager to provide a Compositing Overlay Window (COW). Largely designed around the Texture From Pixmap (TFP) OpenGL extension, the X11 Pixmap is treated as an OpenGL Texture. Not necessary for compositing (KWin4 doesn't use it), but a straightforward way to accomplish it. The reference use of this extension was xcompmgr, but it is subsumed directly into modern window managers such as Compiz and Unity.
• Pixman -- An archive of low-level rasterizing code used by Xorg and Cairo.
• Memory Managers
  • GEM - Graphics Execution Manager. Intel's solution for region and context management.
  • TTM - Transition Table Maps.
• Accelerators
  • XAA - The XFree86 Acceleration Architecture. Added in XFree86 3.3, rewritten entirely for 4.0, and removed in X.org 1.13.
  • KAA - KDrive Acceleration Architecture.
  • EXA - A 3D version of XAA.
  • UXA - Intel's EXA rewrite atop GEM.
  • Xgl - X implemented atop the OpenGL wrapper API Glitz.
    • Xglx -
    • Xegl -
  • AIGLX - Accelerated Indirect GLX. Replaced Xgl in 2008. Present but largely unused today.
  • SNA - Sandy Bridge New Acceleration. Intel's new accelerator, which supports more than Sandy Bridge, but does not support non-Intel chipsets.

Wayland is not yet ready for general use.

Wayland architecture

Wayland

Wayland is a new display/interface protocol, under heavy development as of the end of 2012. It integrates compositing with window management, and eliminates the need for an explicit server process.

APIs

Xorg

Neko, the XCB mascot.

• XCB

The X protocol C language Bindings are a low-level API driving the minimalist XCB X wire protocol. XCB is easily used in an asynchronous fashion.

• • There are no native GLX (X11 OpenGL) bindings implemented atop XCB.
  • There does not appear to be XInput2 support implemented atop XCB.
• X11

The original wire protocol for X11. It is now implemented atop XCB as a series of request+spin+reply kernels.

• Xlib

The original, archaic, extraordinarily ugly X widget set, plus window control routines and other primitives used by higher-level toolkits.

OpenGL

OpenGL 4.3 was specified 2012-08-06.

Mesa 9.0.2 was released on 2013-01-22.

OpenGL (the Open Graphics Library) is an API specification geared towards hardware-accelerated 3D primitives. On desktop Linux, there are three major implementations of OpenGL:

• NVIDIA's proprietary implementation, which supports most NVIDIA cards
• AMD's proprietary implementation (fglrx), which supports most AMD cards
• Mesa, an open implementation mixing hardware acceleration with software fallback. It includes:
  • Gallium, an infrastructure designed to facilitate development of hardware-accelerated drivers
  • TGSI, an intermediate representation used by Gallium to target state trackers
  • Nouveau, an open implementation for NVIDIA cards, using Gallium
  • Radeon, an open implementation for AMD cards, using Gallium
  • An open implementation for Intel cards
  • llvmpipe, a software rasterizer

Generally, the proprietary drivers tend to support newer OpenGL features and provide better performance, while the Mesa drivers are more smoothly integrated with the rest of the graphics stack (Kernel Mode Switching, Direct Rendering Infrastructure, XRandr, etc). Standard Linux tools might fail to work with the proprietary drivers, forcing use of proprietary tools to control the device.

EGL 1.4 was specified 2011-04-09.

• EGL

EGL provides window manager-specific functionality for an OpenGL stack.

OpenGL ES 3.0 was specified 2012-08-06.

• OpenGL ES

OpenGL for Embedded Systems. Mesa targets this with drivers for:

• • OMAP (Texas Instruments)
  • SGX (PowerVR)

OpenVG 1.1 was specified 2008-12-09.

• OpenVG

Vector graphics acceleration targeting embedded systems (ie, often used in conjunction with OpenGL ES).

• GLU/GLUT

The OpenGL Utility Toolkit. These libraries facilitate developing simple OpenGL applications.

• GLEW

The OpenGL Extensions Wrangler. A cross-platform library for determining available OpenGL extension support.

SDL

The Simple DirectMedia Layer is a cross-platform, high-level multimedia library. It supports backends including:

• DirectFB
• Xlib
• OpenGL

DirectFB

A thin layer providing hardware acceleration and graphics primitives atop Linux framebuffer devices.

Video Playback

Hardware

Hardware-offloaded solutions generally do not provide the quality or feature set possible through CPU-based transformations.

• PureVideo -- NVIDIA GPU hardware solution in four Feature Sets.
• Avivo -- AMD GPU hardware solution
• Quick Sync -- Intel CPU hardware solution
• Clear Video -- Intel GPU hardware solution

Software

• VAAPI -- The Video Acceleration API. A cross-platform, open video offloading library.
• VDPAU -- NVIDIA's Video Decode and Presentation API for Unix. It is a superset of VAAPI, and suitable for use as a VAAPI backend.

GNOME + OpenType

• Cairo -- Cairo is a vector drawing library used by applications either directly or through the GTK+ widget set. It's used heavily throughout the GNOME Project. Cairo has multiple backends, including:
  • XCB and Xlib
  • OpenGL and OpenGL ES
  • OpenVG
  • PDF
• FontConfig -- Used to select fonts based on desired properties and complex selectors that virtually no one anywhere understands.
• FreeType -- Font rasterization of non-complex (aka Level One) Unicode glyphs.
• FriBidi -- GNU's implementation of the Unicode Bidirectional Algorithm, necessary for bidirectional and boustrophedonic writing systems.
• Harfbuzz -- An OpenType layout engine capable of complex Unicode phonetic transformations.
• Pango -- A very high-level API for rendering glyphs.

KDE

FIXME!

Tools

Xorg

xrandr controls and accesses the RandR extension:

Screen 0: minimum 8 x 8, current 2560 x 1600, maximum 16384 x 16384
DVI-I-0 disconnected (normal left inverted right x axis y axis)
DVI-I-1 disconnected (normal left inverted right x axis y axis)
DVI-I-2 connected 2560x1600+0+0 (normal left inverted right x axis y axis) 641mm x 401mm
   2560x1600      60.0*+
   1920x1440      60.0  
   1920x1200      59.9  
   1600x1200      60.0  
   1280x1024      75.0     60.0  
   1280x800       59.8  
   1152x864       75.0  
   1024x768       75.0     60.0  
   800x600        75.0     60.3  
   640x480        75.0     59.9  
HDMI-0 disconnected (normal left inverted right x axis y axis)
DVI-I-3 disconnected (normal left inverted right x axis y axis)

xwininfo dumps information about an X window:

xwininfo: Window id: 0x3c00080 "xorg data rtypes - Google Search - Iceweasel"

  Absolute upper-left X:  6
  Absolute upper-left Y:  21
  Relative upper-left X:  6
  Relative upper-left Y:  21
  Width: 2548
  Height: 1573
  Depth: 24
  Visual: 0x21
  Visual Class: TrueColor
  Border width: 0
  Class: InputOutput
  Colormap: 0x20 (installed)
  Bit Gravity State: NorthWestGravity
  Window Gravity State: NorthWestGravity
  Backing Store State: NotUseful
  Save Under State: no
  Map State: IsViewable
  Override Redirect State: no
  Corners:  +6+21  -6+21  -6-6  +6-6
  -geometry 2548x1573+6-6

Framebuffer

• fbset accesses and controls framebuffer devices.
• fbterm provides a terminal on a framebuffer device.
• fbcat and fbgrab can screenshot a framebuffer.

See Also

• VGA compatible text mode on Wikipedia
• "The Linux Graphics Stack"
• cairo-ickle
• carl's boring blog (Carl Worth)
• keithp.com/blog (Keith Packard)
• X Window System Technical Glossary
• "The Art and Science of Text Mode Conversion"
• Fonts in the SprezzOS Manual

{{#switch:|subgroup|child=|none=|#default=

}}{{#ifeq:|Template|{{#ifeq:|child||{{#ifeq:|subgroup||{{#switch:graphics

|doc
|sandbox
|testcases =
|#default = {{#switch:
 |plainlist
 |hlist
 |hlist hnum
 |hlist vcard
 |vcard hlist = 
 |#default = hlist
 }}
}}

}}}}}}