Etnaviv on Christian Gmeiner

The Diagonal Seam

Fri, 19 Jun 2026 00:00:00 +0000

Two more dEQP tests down:

dEQP-GLES3.functional.fbo.blit.rect.nearest_consistency_mag -- Pass
dEQP-GLES3.functional.fbo.blit.rect.nearest_consistency_min -- Pass

This one was a fun geometry puzzle.

The problem

Mesa’s u_blitter is the utility that drivers use for framebuffer blits – copying pixel data between surfaces, optionally scaling and filtering. It works by drawing a textured quad: set up the source as a texture, the destination as a render target, and draw a rectangle with the appropriate texture coordinates. Simple.

Except the quad is made of two triangles. And that diagonal seam between them is where the trouble starts.

 v2 -------- v3
  |  \   T2  |
  | T1  \    |
  |       \  |
 v0 -------- v1

For LINEAR filtering, this is fine – the interpolation across the seam is smooth enough that nobody notices. But for NEAREST filtering, a texel is selected based on which texel center is closest to the interpolated texture coordinate. At the diagonal seam, the two triangles can produce slightly different texture coordinates for pixels that sit right on the boundary. Different coordinates mean different nearest-texel selection, and that means an inconsistent stripe of wrong texels running diagonally across the blit.

The dEQP nearest_consistency tests specifically check for this: they blit with NEAREST filtering and verify that every pixel picks the same texel regardless of which triangle it fell into.

What others do

This isn’t a new problem. V3D already has a workaround in u_blitter: it sets use_index_buffer to reorder the triangle indices so that the shared edge of the two triangles is along a different diagonal. This changes which pixels land on the seam and can be enough to pass the tests on some hardware.

On GC7000, that wasn’t sufficient. The floating-point interpolation differences are large enough that the seam remains visible regardless of which diagonal you pick.

What the blob does

Looking at command stream traces from the proprietary Vivante driver, the answer was clear: they don’t draw a quad at all. They draw a single oversized triangle and let the scissor clip it to the destination rectangle.

No seam, no problem.

The single triangle transform

The idea is simple: take a rectangle and find a single triangle that fully covers it. The smallest such triangle is a right triangle with legs twice the width and twice the height of the rectangle:

 v2
  |\
  |  \
  | rect\
  |------+\
  |      |  \
 v0 -----|--- v1

Vertex 0 stays at the rectangle’s top-left corner. Vertex 1 extends to twice the rectangle width. Vertex 2 extends to twice the rectangle height. The original rectangle is fully contained within this triangle.

The math for transforming the blitter’s 4-vertex quad into a 3-vertex triangle is:

for (unsigned a = 0; a < 2; a++) {      /* pos and texcoord */
   for (unsigned c = 0; c < 4; c++) {   /* xyzw components */
      float v0 = ctx->vertices[0][a][c];
      ctx->vertices[1][a][c] = 2.0f * ctx->vertices[1][a][c] - v0;
      ctx->vertices[2][a][c] = 2.0f * ctx->vertices[3][a][c] - v0;
   }
}

This transforms both position and texture coordinates consistently, so the texture mapping across the visible (scissored) region is identical to what the full quad would have produced – minus the seam.

Scissor is essential

The oversized triangle extends beyond the destination rectangle, so we need scissor to clip it. The blitter doesn’t always have a scissor set up, so when use_single_triangle is enabled, we synthesize one from the destination box:

if (ctx->base.use_single_triangle && !scissor) {
   synth_scissor.minx = MAX2(dstbox->x, 0);
   synth_scissor.miny = MAX2(dstbox->y, 0);
   synth_scissor.maxx = dstbox->x + dstbox->width;
   synth_scissor.maxy = dstbox->y + dstbox->height;
   scissor = &synth_scissor;
}

Keeping it scoped

The single-triangle transform should only apply to blit operations, not to clears or other blitter draws. A transient single_triangle_active flag is set around the actual blit draw calls and checked in the vertex emission code. Drivers opt in by setting use_single_triangle on the blitter context at creation time.

For etnaviv, that’s a single line:

ctx->blitter->use_single_triangle = true;

The design

The implementation is split into two commits: the u_blitter.c infrastructure (vertex transform, synthesized scissor, gating flag, 3-vertex draw path) that any driver can opt into, and the one-line etnaviv enablement. Keeping them separate means another driver hitting the same seam can flip the flag without touching u_blitter code.

The work landed upstream in u_blitter: Add single-triangle draw mode for NEAREST blit consistency.

The takeaway

Sometimes the fix for a rendering artifact isn’t better math or tighter tolerances – it’s removing the geometric feature that causes the problem in the first place. Two triangles have a seam. One triangle doesn’t.

Fixing the R/B swap the right way

Wed, 10 Jun 2026 00:00:00 +0000

If you’ve ever looked at a GPU render and seen blue where red should be, you’ve met the R/B swap problem. For etnaviv this has been a long-standing source of complexity. We were solving it in the shader, but the proprietary blob driver had a simpler approach all along. As part of my work at Igalia, I finally sat down and did it properly.

The problem

Vivante GPUs have a quirk: the Pixel Engine (PE) always writes pixels in BGRA byte order. When your API says “render to R8G8B8A8_UNORM”, what actually lands in memory is B, G, R, A. Every byte of every pixel, every frame. The hardware just works that way.

The question is: where do you fix it?

The etnaviv driver was doing it in the shader. Before the fragment shader writes its output, a NIR lowering pass swaps the R and B channels:

   alu->src[0].swizzle[0] = 2;   /* .r reads from .b */
   alu->src[0].swizzle[2] = 0;   /* .b reads from .r */

This works, until it doesn’t. The shader key needs a frag_rb_swap bitmask per render target. The blend color needs per-RT R/B swapping to match. And it falls apart entirely for scalar outputs - if a shader writes a single float, there’s no .z component to swizzle into .x. That’s exactly the NIR validation failure we hit:

Test case 'dEQP-GLES3.functional.fragment_out.basic.fixed.rgb8_lowp_float'..
NIR validation failed after etna_lower_io in ../mesa/src/gallium/drivers/etnaviv/etnaviv_compiler_nir.c:1296
1 errors:
shader: MESA_SHADER_FRAGMENT
source_blake3: {0x4d463d73, 0x4b27d742, 0x27a92b64, 0x375c010f, 0xb2ce3767, 0x2adc55cc, 0x6da8105b, 0x5b9fce29}
name: GLSL1
prev_stage: MESA_SHADER_VERTEX
inputs_read: 32
outputs_written: 4
perspective_varyings: 32
max_subgroup_size: 128
min_subgroup_size: 1
api_subgroup_size_draw_uniform: true
first_ubo_is_default_ubo: true
known_interpolation_qualifiers: true
flrp_lowered: true
inputs: 1
outputs: 1
decl_var shader_in INTERP_MODE_SMOOTH none highp float packed:var0 (VARYING_SLOT_VAR0.x, 0, 0)
decl_var shader_out INTERP_MODE_NONE none mediump float out0 (FRAG_RESULT_DATA0.x, 0, 0)
decl_function main () (entrypoint)

impl main {
    block b0:  // preds:
    32    %3 = load_const (0x00000000)
    32    %4 = @load_input (%3 (0x0)) (base=0, range=1, component=0, dest_type=float32, io location=VARYING_SLOT_VAR0 slots=1)  // packed:var0
    32    %2 = deref_var &out0 (shader_out mediump float)
    32    %5 = mov %4.z
error: src->swizzle[i] < num_components (../mesa/src/compiler/nir/nir_validate.c:217)

               @store_deref (%2, %5) (wrmask=x, access=none)
               // succs: b1
    block b1:
}

FATAL ERROR: Test program crashed

What the blob does

Looking at command stream traces from the proprietary driver, the answer is almost disappointingly simple. Instead of this:

  Texture format: A8B8G8R8     (read BGRA as BGRA)
  PE format:      A8B8G8R8     (write BGRA)
  Shader:         swap R <-> B

The blob does this:

  Texture format: A8R8G8B8     (read BGRA as RGBA - hardware swaps on read)
  PE format:      A8B8G8R8     (write BGRA - unchanged)
  Shader:         nothing

That’s it. Tell the texture sampler the data is A8R8G8B8, and it will correctly interpret the BGRA bytes as RGBA channels. The PE keeps writing BGRA because that’s what it does. No shader modification needed.

In our format table, the change is a single field:

  -  VT(R8G8B8A8_UNORM, UNSIGNED_BYTE, A8B8G8R8, A8B8G8R8)
  +  VT(R8G8B8A8_UNORM, UNSIGNED_BYTE, A8R8G8B8, A8B8G8R8)
                                        ^^^^^^^^
                                     texture format

The data flow

To understand why this works, trace a red pixel through the pipeline:

                          BGRA-internal byte order
                         +------------------------+
                         |                        |
  API: glClear(1,0,0,1)  |   Memory: [0,0,255,255]|   CPU: expects [255,0,0,255]
  "red = 1.0"            |   (B=0, G=0, R=255,    |   "RGBA order"
                         |    A=255)              |
                         +------------------------+

  +-----------+     +--------+     +--------+     +---------+
  | Shader    |     | PE     |     | Memory |     | Sampler |
  | out=RGBA  | --> | writes | --> | stores | --> | reads   |
  | (1,0,0,1) |     | BGRA   |     | BGRA   |     | as      |
  |           |     |        |     | bytes  |     | A8R8G8B8|
  +-----------+     +--------+     +--------+     +---------+
       |                |               |              |
    R=1.0            B=0x00          [00 00 FF FF]   R=1.0
    G=0.0            G=0x00                          G=0.0
    B=0.0            R=0xFF                          B=0.0
    A=1.0            A=0xFF                          A=1.0

The shader writes (1,0,0,1). The PE swaps R/B on write, so memory gets [0,0,255,255] in BGRA order. The sampler, told the format is A8R8G8B8, reads those same bytes back as (1,0,0,1). Round-trip complete, no shader involvement.

The CPU boundary problem

GPU-to-GPU is clean. But what happens at the CPU boundary - glReadPixels, glTexSubImage, glBlitFramebuffer to a CPU-mapped buffer? The CPU expects RGBA byte order. Memory has BGRA. Something needs to swap.

This is where the hardware copy/resolve engines come in - RS and BLT. Both can perform R/B swapping during their copy operations. The RS engine has a swap_rb bit. The BLT engine has per-side swizzle fields. We just need to activate this at the right moment.

The key insight: only transfer blits (tiled-to-linear copies for CPU access) need the swap. GPU-internal blits - glBlitFramebuffer between two render targets, TS resolve, mipmap generation - are all operating on data already in BGRA order on both sides. Swapping there would be wrong.

So we gate it with a context flag:

ctx->in_transfer_blit = true;
etna_copy_resource_box(pctx, trans->rsc, &rsc->base, ...);
ctx->in_transfer_blit = false;

And in the RS blit path:

.swap_rb = ctx->in_transfer_blit &&
           translate_pe_format_rb_swap(blit_info->src.format),

The texture shadow trap

With the basic approach working, tests passed on GC7000 (BLT engine). But GC2000 (RS engine) had a regression: fbo-blit showed blue where red should be.

After adding debug prints and tracing the code paths, the culprit was the texture shadow. Some resources can’t be sampled directly by the texture unit - for example, a render target might use a layout the sampler doesn’t understand. For these, the driver allocates a second copy of the resource in a sampler-compatible tiled layout. This is the “texture shadow”.

The shadow is a workaround that hurts performance - it means extra memory and extra copies. Ideally we wouldn’t need it at all. But while it exists, the driver uses it as a shortcut for CPU transfers: read directly from the shadow and detile in software, skipping the blit engine:

Passing probes:  PATH=temp_resource+RS_blit    swap_rb=1   -> correct
Failing probes:  PATH=texture_shadow                       -> R/B swapped

The texture shadow path does a software detile - raw byte copy, no R/B swap. With BGRA-internal byte order, that gives you BGRA bytes on the CPU side. Wrong.

The fix: skip the texture shadow shortcut for formats that need R/B swap, forcing through the blit engine path which handles the conversion:

if (rsc->texture && !etna_resource_newer(rsc, etna_resource(rsc->texture)) &&
    !translate_pe_format_rb_swap(prsc->format)) {
   /* Use texture shadow - safe, no R/B swap needed */
   rsc = etna_resource(rsc->texture);
} else {
   /* Use blit engine - handles R/B swap correctly */
   ...
}

Results

This is a net-negative patch series - 15 files changed, 76 insertions, 113 deletions. The etnaviv: Remove RB swap logic in the fragment shader contains all that’s needed:

blt: Use img->swizzle for CONFIG SWIZ fields - preparation for per-image swizzle support
Add translate_pe_internal_format helper - maps RGBA pipe formats to BGRA equivalents for clear color packing
Use BGRA-internal texture format with BLT/RS R/B swizzle - the main change
Compute blend color directly in etna_set_blend_color - simplifies blend color, no deferred update needed

Fixes the NIR validation failure with scalar fragment outputs. And as a nice side effect, removing the shader-based swap means fewer shader variants, fewer instructions and less overhead. glmark2-es2-wayland improves from ~835 to ~874 FPS - a 4.7% performance increase.

Sometimes matching what the hardware vendor does is the right answer. The blob driver figured this out years ago. We just needed to look at the traces.

That’s where I thought the story ended

The texture-format trick has a hidden assumption baked into it: that the GPU both writes and reads every resource. The PE writes BGRA, the sampler is told the format is A8R8G8B8, and the byte order cancels out. It’s a closed loop, and as long as the data never leaves the GPU, nobody outside ever sees the BGRA bytes.

dmabuf breaks the loop.

When a buffer is shared with another process - a Wayland compositor, a video decoder, a camera - the byte order is no longer our private business. It’s mandated by the DRM FourCC. An external producer writes honest RGBA bytes into the buffer. Then our sampler, still convinced the format is A8R8G8B8, reads them as BGRA. Red and blue swap. And on the way out, a transfer blit happily swaps data that was already correct. The optimization that made GPU-internal rendering clean made buffer sharing wrong.

  GPU-internal (closed loop):       dmabuf (loop broken):

  PE writes BGRA                    external producer writes RGBA
       |                                  |
  sampler reads as A8R8G8B8         sampler reads as A8R8G8B8
       |                                  |
  cancels out -> correct            reads RGBA as BGRA -> swapped

Step back out for shared resources

The first fix is the obvious one: when a resource is shared, don’t play the trick. Use the native A8B8G8R8 texture format so the sampler reads RGBA bytes as RGBA, skip the R/B swizzle in the BLT and RS transfer blits, and re-enable the texture shadow shortcut that the swizzle had forced us to disable. Internal resources keep the BGRA-internal optimization untouched.

That handles imports. But there’s a case it doesn’t cover: a resource we rendered into and then export. The PE wrote BGRA, because that’s all the PE knows how to do. The external consumer expects native order.

Here a second kind of shadow shows up. Just as the sampler gets a texture shadow when it can’t read the base layout, the PE gets a render shadow - a render-compatible copy - when it can’t draw into the base layout. (On some GPUs, like GC2000, no single tiling satisfies both the texture engine and the pixel engine, so a resource can end up carrying both shadows.) When an exported resource is flushed, that render shadow is resolved back to the base. So we hook etna_flush_resource() and do the R/B swap during that copy - using the BLT destination swizzle or the RS SWAP_RB bit. The swap rides along on a copy we were doing anyway.

One buffer, two byte orders

Now the same shared buffer can be in one of two states. Just imported, or just flushed for export? Native RGBA. Freshly rendered by the PE, not yet flushed? PE-internal BGRA. A static texture format chosen at sampler-view creation can’t be right for both.

So the format choice becomes dynamic. A shared_native_order flag tracks which order the bytes are currently in, and the sampler-view format follows from it:

`shared_native_order`	How you get there	Bytes in buffer	Sampler format
`true`	set on import, and again after `flush_resource()` swaps on export	native RGBA	`A8B8G8R8` (native)
`false`	cleared when the PE renders straight into the buffer with no render shadow, and no shader swap fixed up the bytes (see below)	PE-internal BGRA	`A8R8G8B8` (the trick)

Both texture paths - state-based and descriptor-based - pre-compute the native format variant at sampler-view creation, so picking the right one at emit time costs a single branch, not a per-frame format recompute.

The shader swap comes back

Which brings us to LINEAR_PE GPUs, and a twist I didn’t see coming.

On these GPUs, a linear shared resource is render-compatible. There is no render shadow - the PE writes straight into the buffer that gets handed to the compositor. So after a draw, the buffer holds BGRA, and flush_resource() would have to issue a full-surface blit to swap it. That’s real bandwidth, on every frame, that the old shadow-based path never paid.

There’s a cheaper place to do the swap: in the shader, on the way out. Which is exactly the thing this whole series set out to delete.

So it comes back - but only for this one case, and done properly. A per-RT frag_rb_swap bitmask in the shader key drives a NIR lowering pass that swaps channels 0 and 2 on the fragment output. The original shader swap fell over on scalar outputs, because there was no .z to swizzle from. This one widens the output variable to vec4 first, padding the missing components with undef, then applies the swizzle with an adjusted writemask:

   /* Pad source to 4 components (undef for missing) */
   nir_def *padded = nir_pad_vec4(&b, src);

   /* Swap R and B channels */
   unsigned swiz[] = {2, 1, 0, 3};

That’s the scalar rgb8_lowp_float crash from the top of this post - fixed, in the one path that now needs a shader swap at all.

Wiring it up is a check in etna_draw_vbo():

   if (VIV_FEATURE(screen, ETNA_FEATURE_LINEAR_PE)) {
      for (i = 0; i < pfb->nr_cbufs; i++) {
         struct etna_resource *rsc = etna_resource(pfb->cbufs[i].texture);
         if (rsc->shared && rsc->layout == ETNA_LAYOUT_LINEAR &&
             translate_pe_format_rb_swap(pfb->cbufs[i].format))
            key.frag_rb_swap |= (1 << i);
      }
   }

The bitmask is per-RT, so an MRT setup with a mix of shared and private targets does the right thing for each. And because the shader produced native bytes directly, shared_native_order stays true and flush_resource() skips its blit entirely.

The fixes for all of this live in a follow-up series, etnaviv: Fix dmabuf R/B byte order for PE_FORMAT_RB_SWAP formats.

So was removing the shader swap a mistake?

No - but it wasn’t the whole answer either.

The shader swap was wrong as the universal solution. It cost a shader-key dimension, per-RT blend-color fixups, and it crashed on scalar outputs. The texture-format trick is genuinely better for the common case, where a resource lives and dies on the GPU.

What the dmabuf work showed is that there is no single right place to fix R/B order. There’s a place that’s cheapest for each path: the texture format for GPU-internal resources, a transfer-blit swap at the CPU boundary, a flush-time swap for exported render targets, and - for LINEAR_PE, where the PE writes straight into a shared buffer - the shader, after all. The trick isn’t picking one. It’s knowing which boundary you’re standing on, and swapping there.

Multiple Render Targets for etnaviv

Thu, 16 Jan 2025 00:00:00 +0000

Modern graphics programming revolves around achieving high-performance rendering and visually stunning effects. Among OpenGL’s capabilities, Multiple Render Targets (MRTs) are particularly valuable for enabling advanced rendering techniques with greater efficiency.

With the latest release of Mesa 24.03 and the commitment from Igalia, the etnaviv GPU driver now includes support for MRTs. If you’ve ever wondered how MRTs can transform your graphics pipeline or are curious about the challenges of implementing this feature, this blog post is for you.

Understanding Multiple Render Targets (MRTs)

At its core, MRTs allow rendering to multiple images or “render targets” simultaneously during a single rendering pass. These render targets are buffers or textures that store various scene data such as color, depth, or normals. By writing to multiple targets at once, MRTs enable developers to:

Enhance efficiency by reducing the number of rendering passes.
Implement sophisticated rendering techniques, such as deferred shading, which decouples geometry processing from lighting and shading.

Here’s a simple OpenGLES shader example demonstrating how to use MRTs:

// Vertex Shader
layout(location = 0) in vec2 inPosition;

void main() {
    gl_Position = vec4(inPosition, 0.0, 1.0);
}

// Fragment Shader
layout(location = 0) out vec4 fragColor1;
layout(location = 1) out vec4 fragColor2;

void main() {
    fragColor1 = vec4(1.0, 0.0, 0.0, 1.0); // Red
    fragColor2 = vec4(0.0, 0.0, 1.0, 1.0); // Blue
}

This code renders two render targets: one red and one blue, by declaring two output locations in the fragment shader and writing different colors to each of them.

How MRTs Are Used

MRTs play an important role in graphics applications. Here are some of their most common use cases:

1. Deferred Rendering

MRTs are the backbone of deferred shading, a technique where scene geometry is rendered to multiple targets, storing data like positions, normals, and albedo. This data is then processed in a second pass to calculate lighting, enabling advanced effects like dynamic shadows and screen-space reflections.

2. Post-Processing Effects

By writing intermediate results to multiple targets, MRTs facilitate a wide range of effects, including bloom, depth of field, and ambient occlusion.

3. Debugging and Visualization

Developers can output different types of scene data simultaneously for debugging or creating visualizations.

Implementing MRTs in etnaviv

Adding MRT support to the etnaviv driver involved significant reverse engineering and experimentation. Here’s a look at some of the challenges and solutions:

Reverse Engineering the GPU

The reverse engineering process begins with examining the limits exposed by the binary blob driver, such as the value returned by GL_MAX_DRAW_BUFFERS. Different Vivante GPU generations (HALTI) support varying numbers of render targets, requiring repeated testing and analysis.

Lay the foundation

I started with Freedreno’s test-mrt-fbo to get a rought idea of what needed to be done. Hours later, I identified most of the necessary bits and GPU states I had seen in the command stream dumps generated by the proprietary driver.

The first goal was to get a very basic piglit MRT test working on the GC7000 (HALTI5) GPU, rendering correctly with etnaviv. In this phase of reverse engineering, I usually hack around in the driver and my git commit history is full of ‘hack/wip’ commits. This helps me keep track of the (breaking) changes I make during the process.

Looking at the changes I had made until here, it was clear that I had touched nearly every part of the gallium driver. It started with compiler changes to handle the extra color outputs and ended with the extra MRT states that needed to be emitted. And, to be honest, at this stage I also had broken some other CTS and piglit tests, but that was something to be taken care of later.

When I tried some more complex piglit tests that used sparse render targets, I saw that they failed and realized, after reverse-engineering the proprietary driver some more, that I needed to work on remapping some information provided by NIR in Mesa about fragment shader outputs into compressed info the driver needed. That didn’t make all relevant Piglit tests pass, but I was getting close.

Will it work on another GPU?

The branch was then tested on older Vivante GPUs, like the GC3000 (HALTI2). None of the tests passed initially, as state emissions differed and the maximum MRT count was lower (4 vs. 8). Updating the state emission logic addressed these issues.

Whats wrong with the Tile Status?

As I always want to provide the best experience for all etnaviv users, I wanted to support this shiny new feature even on a much older Vivante GPU generation - GC2000 (HALT0) found in i.MX6 boards.

My hopes where high that my git branch would just work but, as usual, I was wrong. The traces from the vendor driver showed something interesting. There was no tile status (TS) usage found in the traces and I could confirm piglit was happy when I used ETNA_MESA_DEBUG=no_ts.

You might wonder that the ominous Tile Status might be. Let me give you a quick summary.

A render target is divided in tiles, and every tile has a couple of status flags. An auxiliary buffer - the so called Tile Status buffer - associated with each render surface keeps track of these tile status flags. One of these flags is the clear flag, that signifies that the tile has been cleared. For example, a fast clear happens by setting the clear bit for each tile instead of clearing the actual surface data.

I found a way to fix this problem too and then moved to CI.

CI to test them all

If you’ve followed me until here, you know that I worked on one HALTI and moved to another and never tested if I had broken any of the other HALTI’s. It was time to let etnaviv’s CI do its work and, as expected, I discovered I had broken HALTI5. After a local debugging session I got it into a working state. While I was very close to being ready to submit an MR and incorporate review feedback, there was one last thing I need to take care of…

HALTI5+ Enhancements for MRTs

It turns out that a HALTI5 GPU can support more OpenGL extensions as it has even more GPU states that fall under the MRT umbrella. The MRT ground work that was already in place allowed me to implement the following ones:

1. GL_EXT_draw_buffers2

This extension allows independent blending and write masking for each render target in an MRT setup. Developers can specify unique blending equations and masks for each target, offering unparalleled control over how data is combined.

2. GL_ARB_draw_buffers_blend

This feature extends blending capabilities even further, enabling per-buffer blending equations and functions. It’s especially useful for advanced rendering pipelines, such as deferred shading and post-processing, where different render targets may require entirely distinct blending behaviors.

Conclusion

With the addition of MRT support and the powerful HALTI5+ enhancements like GL_EXT_draw_buffers2 and GL_ARB_draw_buffers_blend, the etnaviv driver has reached a significant milestone.

MRT support is also a key feature for achieving full GLES3 compliance, marking a step forward in modernizing the capabilities of the etnaviv driver.

For detailed reverse engineering results, check out this repository, which uses the rnndb format to describe GPU states and bits. The MRT specific changes can be found in this commit.

What’s next? Stay tuned for more updates on the etnaviv driver as we continue to push more features upstream.

CI-Tron: A Long Road to a Better Board Farm

Wed, 30 Oct 2024 00:00:00 +0000

I’m a big supporter of finding problems before they get into the code base. The earlier you catch issues, the easier they are to fix. One of the main tools that helps with this is a Continuous Integration (CI) farm. A CI farm allows you to run extensive tests like deqp or piglit on a merge request or even on a private git branch before any code is merged, which significantly helps catch problems early.

I’m not the first one at Igalia to think this is really important. We already have a large Raspberry Pi board farm available on freedesktop’s GitLab instance that serves as a powerful tool for validating changes before they hit the main branch.

For a while, however, the etnaviv board farm has been offline. The main reason? I needed to clean up the setup: re-house it in a proper rack, redo all the wiring, and add more devices. What initially seemed like a few days’ worth of work spiraled into months of delay, mostly because I wanted to transition to using ci-tron.

Getting Familiar with the Ci-Tron Setup

Before diving into my journey, let’s quickly cover what makes up a ci-tron board farm.

Ci-Tron Gateway: This component is the central hub that manages devices.
PDU (Power Delivery Unit): A PDU is a device that manages the electrical power distribution to all the components in the CI farm. It allows you to remotely control the power, including power cycling devices, which is crucial for automating device management.
DUT (Device Under Test): The heart of the CI farm—these are the devices where the actual testing happens.

The Long Road to a Working Farm

Over the past few months, I’ve been slowly preparing for the big ci-tron transition. The first step was ensuring my PDU was compatible. It wasn’t initially supported, but after some hacking, I got it working and submitted a merge request (MR). After a few rounds of revisions, it was merged, expanding ci-tron’s PDU support significantly.

The next and most critical step was getting a DUT to boot up correctly. Initially, ci-tron only supported iPXE as a boot method, but my devices are using U-Boot. I tried to make it work anyway, but the network initialization failed too often, and I found myself sinking hours into debugging.

Thankfully, rudimentary support for a U-Boot based boot flow was eventually added. After some tweaks, I managed to get my DUTs booting — but not without complications. A major problem was getting the correct Device Tree Blob (DTB) to load, which was needed for ci-tron’s training rounds. A Device Tree Blob (DTB) is a binary representation of the hardware layout of a device. The DTB is used by the Linux kernel to understand the hardware configuration, including components like the CPU, memory, and peripherals. In my case, ensuring that the correct DTB was provided was crucial for the DUT to boot and be correctly managed by ci-tron. While integrating the DTB into U-Boot was suggested, it wasn’t ideal. Updating the bootloader just to change a DTB is cumbersome, especially with multiple devices in the farm.

With the booting issue taking up too much time, I decided to put it on hold and focus on something else: gfxinfo.

Gfxinfo Integration Challenges

gfxinfo is a neat feature that automatically tags a DUT based on the GPU model in the system, avoiding the need for manually assigning tags like gc2000. In theory, it’s very convenient—but in practice, there were hurdles.

gfxinfo tags Vivante GPUs using the device tree node information. However, since Vivante GPUs are quite generic, they don’t have a specific model property that uniquely identifies them. The plan was to pull this information using ioctl() calls to the etnaviv kernel driver. It took a lot of back and forth in review due to the internal gfxinfo API being under-documented, but after a lot of effort, I finally got the necessary code merged. You can find all of it in this MR.

Final Push: Getting Everything to Boot

There was still one major obstacle — getting the DUT to boot reliably. Luckily, mupuf was already working on it and made a significant MR with over 80 patches to address the boot issues. Introducing “boots db,” a feature designed to decouple the boot process, granting full control over DHCP, TFTP, and HTTP servers to each job. This is paired with YAML configurations to flexibly define the boot specifics for each board.

As of a few days ago, the latest official ci-tron gateway image contains everything needed to get an etnaviv DUT up and running successfully.

I have to say, I’m very impressed with the end result. It took a lot longer than I had anticipated, but we finally have a plug-and-play CI farm solution for etnaviv. There are still a few missing features—like Network Block Device (NBD) support and some advanced statistics—but the ci-tron team is doing an excellent job, and I’m optimistic about what’s coming next.

Conclusion: A Long Road, but Worth It

The journey to get the etnaviv board farm back online was longer than expected, full of unexpected challenges and technical hurdles. But it was worth it. The result is a robust, automated solution that makes CI testing easier and more reliable for everyone. With ci-tron, it’s easier to find and fix problems before they ever make it into the code base, which is exactly what a good CI setup should be all about. There is still some work to be done on the GitLab side to switch all etnaviv jobs to the new board farm.

If you’re thinking about setting up your own CI farm or migrating to ci-tron, I hope my experience helps smooth the road for you a bit. It might be a long journey, but the end results are absolutely worth it.

It All Started With a Nop - Part I

Thu, 11 Jul 2024 00:00:00 +0000

Note

This blog post is part 1 of a series of blog posts about isaspec and its usage in the etnaviv GPU stack.

I will add here links to the other blog posts, once they are published.

The first time I heard about isaspec, I was blown away by the possibilities it opens. I am really thankful that Igalia made it possible to complete this crucial piece of core infrastructure for the etnaviv GPU stack.

If isaspec is new to you, here is what the Mesa docs have to tell about it:

isaspec provides a mechanism to describe an instruction set in XML, and generate a disassembler and assembler. The intention is to describe the instruction set more formally than hand-coded assembler and disassembler, and better decouple the shader compiler from the underlying instruction encoding to simplify dealing with instruction encoding differences between generations of GPU.

Benefits of a formal ISA description, compared to hand-coded assemblers and disassemblers, include easier detection of new bit combinations that were not seen before in previous generations due to more rigorous description of bits that are expect to be ‘0’ or ‘1’ or ‘x’ (dontcare) and verification that different encodings don’t have conflicting bits (i.e. that the specification cannot result in more than one valid interpretation of any bit pattern).

If you are interested in more details, I highly recommend Rob Clark’s introduction to isaspec presentation.

Target ISA

Vivante uses a fixed-size (128 bits), predictable instruction format with explicit inputs and outputs.

As of today, there are three different encodings seen in the wild:

Base Instruction Set
Extended Instruction Set
Enhanced Vision Instruction Set (EVIS)

Why do I want to switch to isaspec

There are several reasons..

The current state

The current ISA documentation is not very explicit and leaves lot of room for interpretation and speculation. One thing that it provides, are some nice explanations what an instruction does. isaspec does not support tags yet, but I there is a PoC MR that generates really nice looking and information ISA documentation based on the xml.

I think soon you might find all etnaviv’s isaspec documentation at docs.mesa3d.org.

No unit tests

There are no unit tests based on instructions generated by the blob driver. This might not sound too bad, but it opens the door to generating ‘bad’ encoded instructions that could trigger all sorts of weird and hard-to-debug problems. Such breakages could be caused by some compiler rework, etc.

In an ideal world, there would be a unit test that does the following:

Disassembles the binary representation of an instruction from the blob to a string representation.
Verifies that it matches our expectation.
Assembles the string representation back to 128 bits.
Verifies that it matches the binary representation from the blob driver.

This is our ultimate goal, which we really must reach. etnaviv will not be the only driver that does such deep unit testing - e.g. freedreno does it too.

Easier to understand code

Do you remember the rusticl OpenCL attempt for etnaviv? It contains lines like:

      if (nir_src_is_const(intr->src[1])) {
         inst.tex.swiz = 128;
      }

      if (rmode == nir_rounding_mode_rtz)
         inst.tex.amode = 0x4 + INST_ROUND_MODE_RTZ;
      else /*if (rmode == nir_rounding_mode_rtne)*/
         inst.tex.amode = 0x4 + INST_ROUND_MODE_RTNE;

Do you clearly see what is going on? Why do we need to set tex.amode for an ALU instruction?

I always found it quite disappointing to see such code snippets. Sure, they mimic what the blob driver is doing, but you might lose all the knowledge about why these bits are used that way days after you worked on it. There must be a cleaner, more understandable, and thus more maintainable way to document the ISA better.

This situation might become even worse if we want to support the other encodings and could end up with more of these bad patterns, resulting in a maintenance nightmare.

Oh, and if you wonder what happened to OpenCL and etnaviv - I promise there will be an update later this year.

Python opens the door to generate lot of code

As isaspec is written in Python, it is really easy to extend it and add support for new functionality.

At its core, we can generate a disassembler and an assembler based on isaspec. This alone saves us from writing a lot of code that needs to be kept in sync with all the ISA reverse engineering findings that happen over time.

As isaspec is just an ordinary XML file, you can use any programming language you like to work with it.

One source of truth

I really fell in love with the idea of having one source of truth that models our target ISA, contains written documentation, and extends each opcode with meta information that can be used in the upper layers of the compiler stack.

Missing Features

I think I have sold you the idea quite well, so it must be a matter of some days to switch to it. Sadly no, as there are some missing features:

Only max 64 bits width ISAs are supported
Its home in src/freedreno
Alignment support is missing
No tags are supported

Add support for 128 bit wide instructions

The first big MR I worked on, extended BITSET APIs with features needed for isaspec. Here we are talking about bitwise AND, OR, and NOT, and left shifts.

The next step was to switch isaspec to use the BITSET API to support wider ISAs. This resulted in a lot of commits, as there was a need for some new APIs to support handling this new feature. After these 31 commits, we were able to start looking into isaspec support for etnaviv.

Decode Support

Now it is time to start writing an isaspec XML for etnaviv, and the easiest opcode to start with is the nop. As the name suggests, it does nothing and has no src’s, no dst, or any other modifier.

As I do not have this initial version anymore, I tried to recreate it - it might have looked something like this:



 name="#instruction">
	
		{NAME} void, void, void, void
	

	 low="6" high="10">00000
	 pos="11">0

	 pos="12">0
	 low="13" high="26">00000000000000
	 low="27" high="31">00000

	 pos="32">0
	 pos="33">0
	 pos="34">0
	 low="35" high="38">0000
	 pos="39">0
	 low="40" high="42">000

	
	 pos="43">0 
	 low="44" high="52">000000000 
	 pos="53">0
	 low="54" high="61">00000000 
	 pos="62">0 
	 pos="63">0 
	 low="64" high="66">000 
	 low="67" high="69">000 

	
	 pos="70">0 
	 low="71" high="79">000000000 
	 low="81" high="88">00000000 
	 pos="89">0 
	 pos="90">0 
	 low="91" high="93">000 
	 pos="94">0
	 pos="95">0
	 low="96" high="98">000 

	
	 pos="99">0 
	 low="100" high="108">000000000 
	 low="110" high="117">00000000 
	 pos="118">0 
	 pos="119">0 
	 pos="120">0
	 low="121" high="123">000 
	 low="124" high="126">000 
	 pos="127">0





 name="nop" extends="#instruction">
	 low="0" high="5">000000 
	 pos="80">0

With the knowledge of the old ISA documentation, I went fishing for instructions. I only used instructions from the binary blob for this process. It is quite important for me to have as many unit tests as I can write to not break any decoding with some isaspec XML changes I do. And it was a huge lifesaver at that time.

After I reached almost feature parity with the old disassembler, I thought it was time to land etnaviv.xml and replace the current handwritten disassembler with a generated one - yeah, so I submitted an MR to make the switch.

As this is only a driver internal disassembler used by maybe 2-3 human beings, it would not be a problem if there were some regressions.

Today I would say the isaspec disassembler is superior to the handwritten one.

Encode Support

The next item on my list was to add encoding support. As you can imagine, there was some work needed upfront to support ISAs that are bigger than 64 bits. This time the MR only contains two commits 😄.

With everything ready it is time to add isaspec based encoding support to etnaviv.

The goal is to drop our custom (and too simple) assembler and switch to one that is powered by isaspec.

This opens the door to:

Modeling special cases for instructions like a branch with no src’s to a new jump instruction.
Doing the NIR src -> instruction src mapping in isaspec.
Supporting different instruction encodings.
Adding meta information to instructions.

Supporting special instructions that are used in compiler unit tests

In the end, all the magic that is needed is shown in the following diff:

diff --git a/src/etnaviv/isa/etnaviv.xml b/src/etnaviv/isa/etnaviv.xml
index eca8241a2238a..c9a3ebe0a40c2 100644
--- a/src/etnaviv/isa/etnaviv.xml
+++ b/src/etnaviv/isa/etnaviv.xml
@@ -125,6 +125,13 @@ SPDX-License-Identifier: MIT
 	
 	
 	
+
+	
+		p->DST_USE
+		src->amode
+		src->reg
+		p->COMPS
+	
 
 
 
@@ -137,6 +144,46 @@ SPDX-License-Identifier: MIT
 	
 		{TYPE_BIT2} << 2 | {TYPE_BIT01}
 	
+
+	
+		src->type & 0x3
+		(src->type & 0x4) > 2
+		src->sel_bit0
+		src->sel_bit1
+		src->cond
+		src->rounding
+		src->sat
+		src->dst.use
+		&src->dst
+		src->dst_full
+		src->dst.write_mask
+		&src->src[0]
+		src->src[0].use
+		src->src[0].reg
+		src->src[0].rgroup
+		src->src[0].amode
+		&src->src[1]
+		src->src[1].use
+		src->src[1].reg
+		src->src[1].rgroup
+		src->src[1].amode
+		&src->src[2]
+		rc->src[2].use
+		src->src[2].reg
+		src->src[2].rgroup
+		src->src[2].amode
+
+		src->tex.id
+		src->tex.swiz
+		src->imm
+
+		
+		1
+		0
+		0
+		0
+		0
+	
 
 
 
@@ -148,6 +195,13 @@ SPDX-License-Identifier: MIT
 	
 	
 	
+
+	
+		(src & 0x03) >> 0
+		(src & 0x0c) >> 2
+		(src & 0x30) >> 4
+		(src & 0xc0) >> 6
+	
 
 
 
@@ -272,6 +326,13 @@ SPDX-License-Identifier: MIT
 			
 		
 	
+
+	
+		src->swiz
+		src->neg
+		src->abs
+		p->SRC_RGROUP
+

One nice side effect of this work is the removal of isa.xml.h file that has been part of etnaviv since day one. We are able to generate all the file contents with isaspec and some custom python3 scripts. The move of instruction src swizzling from the driver into etnaviv.xml was super easy - less code to maintain!

Summary

I am really happy with the end result, even though it took quite some time from the initial idea to the point when everything was integrated into Mesa’s main git branch.

There is so much more to share - I can’t wait to publish parts II and III.

hwdb - The only truth

Fri, 12 Apr 2024 00:00:00 +0000

Trusting hardware, particularly the registers that describe its functionality, is fundamentally risky.

tl;dr

The etnaviv GPU stack is continuously improving and becoming more robust. This time, a hardware database was incorporated into Mesa, utilizing header files provided by the SoC vendors.

If you are interested in the implementation details, I recommend checking out this Mesa MR.

Are you employed at Versilicon and want to help? You could greatly simplify our work by supplying the community with a comprehensive header that includes all the models you offer.

Last but not least: I deeply appreciate Igalia’s passion for open source GPU driver development, and I am grateful to be a part of the team. Their enthusiasm for open source work not only pushes the boundaries of technology but also builds a strong, collaborative community around it.

The good old days

Years ago, when I began dedicating time to hacking on etnaviv, the kernel driver in use would read a handful of registers and relay the gathered information to the user space blob. This blob driver was then capable of identifying the GPU (including model, revision, etc.), supported features (such as DXT texture compression, seamless cubemaps, etc.), and crucial limits (like the number of registers, number of varyings, and so on).

For reverse engineering purposes, this interface is super useful. Image if you could change one of these feature bits on a target running the binary blob.

With libvivhook it is possible to do exactly this. From time to time, I am running such an old vendor driver stack on an i.MX 6QuadPlus SBC, which features a Vivante GC3000 as its GPU.

Somewhere, I have a collection of scripts that I utilized to acquire additional knowledge about unknown GPU states activated when a specific feature bit was set.

To explore a simple example, let’s consider the case of misrepresenting a GPU’s identity as a GC2000. This involves modifying the information provided by the kernel driver to the user space, making the user space driver believe it is interacting with a GC2000 GPU. This scenario could be used for testing, debugging, or understanding how specific features or optimizations are handled differently across GPU models.

export ETNAVIV_CHIP_MODEL="0x2000"
export ETNAVIV_CHIP_REVISION="0x5108"
export ETNAVIV_FEATURES0_CLEAR="0xFFFFFFFF"
export ETNAVIV_FEATURES1_CLEAR="0xFFFFFFFF"
export ETNAVIV_FEATURES2_CLEAR="0xFFFFFFFF"
export ETNAVIV_FEATURES0_SET="0xe0296cad"
export ETNAVIV_FEATURES1_SET="0xc9799eff"
export ETNAVIV_FEATURES2_SET="0x2efbf2d9"
LD_PRELOAD="/lib/viv_interpose.so" ./test-case

If you capture the generated command stream and compare it with the one produced under the correct identity, you’ll observe many differences. This is super useful - I love it.

Changing Tides: The Shift in ioctl() Interface

At some point in time, Vivante changed their ioctl() interface and modified the gcvHAL_QUERY_CHIP_IDENTITY command. Instead of providing a very detailed chip identity, they reduced the data set to the following values:

model
revision
product id
eco id
customer id

This shift could indeed hinder reverse engineering efforts significantly. At a glance, it becomes impossible to alter any feature value, and understanding how the vendor driver processes these values is out of reach. Determining the function or impact of an unknown feature bit now seems unattainable.

However, the kernel driver also requires a mechanism to verify the existing features of the GPU, as it needs to accommodate a wide variety of GPUs. Therefore, there must be some sort of system or method in place to ensure the kernel driver can effectively manage and support the diverse functionalities and capabilities of different GPUs.

A New Approach: The Hardware Database Dilemma

Let’s welcome: gc_feature_database.h, or hwdb for short.

Vivante transitioned to using a database that stores entries for limit values and feature bits. This database is accessed by querying with model, revision, product id, eco id and customer id.

There is some speculation why this move was done. My theory posits that they became frustrated with the recurring cycle of introducing feature bits to indicate the implementation of a feature, subsequently discovering problems with said feature, and then having to introduce additional feature bits to signal that the feature now truly operates as intended. It became far more straightforward to deactivate a malfunctioning feature by modifying information in the hardware database (hwdb). After they began utilizing the hwdb within the driver, updates to the feature registers in the hardware ceased.

Here is a concrete example of such a case that can be found in the etnaviv gallium driver:

screen->specs.tex_astc = VIV_FEATURE(screen, chipMinorFeatures4, TEXTURE_ASTC) &&
                            !VIV_FEATURE(screen, chipMinorFeatures6, NO_ASTC);

Meanwhile, in the etnaviv world there was a hybrid in the making. We stuck with the detailed feature words and found a smart way to convert from Vivante’s hwdb entries to our own in-kernel database. There is even a full blown Vivante -> etnaviv hwdb convert.

At that time, I did not fully understand all the consequences this approach would bring - more on that later. So, I dedicated my free time to reverse engineering and tweaking the user space driver, while letting the kernel developers do their thing.

About a year after the initial hwdb landed in the kernel, I thought it might be a good idea to read out the extra id values, and provide them via sysfs to the user space. At that time, I already had the idea of moving the hardware database to user space in mind. However, I was preoccupied with other priorities that were higher on my to-do list, and I ended up forgetting about it.

Challange accepted

Tomeu Vizoso began to work on teflon and a Neural Processing Unit (NPU) driver within Mesa, leveraging a significant amount of the existing codebase and concepts, including the same kernel driver for the GPU. During this process, he encountered a need for some NPU-specific limit values. To address this, he added an in-kernel hwdb entry and made the limit values accessible to user space.

That’s it — the kernel supplies all the values the NPU driver requires. We’re finished, aren’t we?

It turns out, that there are many more NPU related values that need to be exposed in the same manner, with seemingly no end in sight.

One of the major drawbacks when the hardware database (hwdb) resides in the kernel is the considerable amount of time it takes for hwdb patches to be written, reviewed, and eventually merged into Linus’s git tree. This significantly slows down the development of user space drivers. For end users, this means they must either run a bleeding-edge kernel or backport the necessary changes on their own.

For me personally, the in-kernel hardware database should never have been implemented in its current form. If I could go back in time, I would have voiced my concerns.

As a result, moving the hardware database (hwdb) to user space quickly became a top priority on my to-do list, and I began working on it. However, during the testing phase of my proof of concept (PoC), I had to pause my work due to a kernel issue that made it unreliable for user space to trust the ID values provided by the kernel. Once my fix for this issue began to be incorporated into stable kernel versions, it was time to finalize the user space hwdb.

There is only one little but important detail we have not talked about yet. There are vendor specific versions of gc_feature_database.h based on different versions of the binary blob. For instance, there is one from NXP, ST, Amlogic and some more.

Here is a brief look at the differences:

nxp/gc_feature_database.h (autogenerated at 2023-10-24 16:06:00, 861 struct members, 27 entries)
stm/gc_feature_database.h (autogenerated at 2022-12-29 11:13:00, 833 struct members, 4 entries)
amlogic/gc_feature_database.h (autogenerated at 2021-04-12 17:20:00, 733 struct members, 8 entries)

We understand that these header files are generated and adhere to a specific structure. Therefore, all we need to do is write an intelligent Python script capable of merging the struct members into a single consolidated struct. This script will also convert the old struct entries to the new format and generate a header file that we can use.

I’m consistently amazed by how swiftly and effortlessly Python can be used for such tasks. Ninety-nine percent of the time, there’s a ready-to-use Python module available, complete with examples and some documentation. To address the C header parsing challenge, I opted for pycparser.

The final outcome is a generated hwdb.h file that looks and feels similar to those generated from the binary blob.

Future proof

This header merging approach offers several advantages:

It simplifies the support for another SoC vendor.
There’s no need to comprehend the significance of each feature bit.
The source header files are supplied by Versilicon or the SoC vendor, ensuring accuracy.
Updating the hwdb is straightforward — simply replace the files and rebuild Mesa.
It allows for much quicker deployment of new features and hwdb updates since no kernel update is required.
This method accelerates the development of user space drivers.

While working on this topic I decided to do a bigger refactoring with the end goal to provide a struct etna_core_info that is located outside of the gallium driver.

This makes the code future proof and moves the filling of struct etna_core_info directly into the lowest layer - libetnaviv_drm (src/etnaviv/drm).

We have not yet talked about one important detail.

What happens if there is no entry in the user space hwdb?

The solution is straightforward: we fallback to the previous method and request all feature words from the kernel driver. However, in an ideal scenario, our user space hardware database should supply all necessary entries. If you find that an entry for your GPU/NPU is missing, please get in touch with me.

What about the in-kernel hwdb?

The existing system, despite its limitations, is set to remain indefinitely, with new entries being added to accommodate new GPUs. Although it will never contain as much information as the user space counterpart, this isn’t necessarily a drawback. For the purposes at hand, only a handful of feature bits are required.

The Year 2023 in Retrospect

Tue, 26 Dec 2023 09:03:20 -0800

Holidays are here and I have time to look back at 2023. For six months I have been working for Igalia and what should I say?

I ❤️ it!

This was the best decision to leave my comfort zone of a normal 9-5 job. I am so proud to work on open source GPU drivers and I am able to spend much of my work time on etnaviv.

Driver maintenance

Before adding any new feature I thought it would be great idea to improve the current state of etnaviv’s gallium driver. Therefor I reworked some general driver code to be more consistent and to have a more modern feeling, and made it possible to drop some hand-rolled conversion helpers by switching to already existing solutions (U_FIXED(..), S_FIXED(..), float_to_ubyte(..)).

I worked through the low hanging fruits of crashes seen in CI runs and fixed many of them.

Feature wise, I also looked at some easy to implement extensions like GL_NV_conditional_render and GL_OES_texture_half_float_linear.

Besides the gallium driver I also worked on some NIR and isaspec features that are beneficial for etnaviv.

XDC2023

A personal highlight was to give a talk about etnaviv at XDC2023 in person.

You might wonder what happened since mid October in etnaviv land.

GLES3

I worked on some features that are needed to expose GLES3 and it turned out that an easy to maintain, extend and test compiler backend is needed. Sadly etnaviv’s current backend compiler does not check any of these boxes. It is so fragile that I only added some needed lowerings to pass some of the dEQP-GLES3.functional.shaders.texture_functions.* tests.

Some more fun work regarding some feature emulation is on the horizon and it’s blocked again by the current compiler.

Backend Compiler

etnaviv includes an isaspec powered disassembler now - a small step towards a new backend compiler. Next on the road to success is the etnaviv backend IR with an assembler.

The new backend compiler is able to run OpenCL kernels with the help of rusticl but I want to land the new backend compiler in smaller chunks that are easier to review.

Multiple Render Targets

During my XDC presentation I talked about a feature I got working on GC7000L - Multiple Render Targets (MRT). At this point it was more or less a proof-of-concept regarding the gallium drivers. There were some missing bits and register for full support on more GPU models and therefore more reverse engineering work was needed. Also the gallium driver needed lots of work to add support for MRT.

Some weeks later I had MRT working on a wider range of Vivante GPUs that are supporting this feature. This includes GC2000, GC3000 and GC7000 models among others. As etnaviv makes heavy use of GPU features it should work on even more models.

Looking forward to 2024

I am really confident that we will see GLES3 and OpenCL for etnaviv. As driver testing is quite important for my work I will expand my current board farm and will look into the new star in CI world - ci-tron.

With that, have a happy holiday season and we’ll be back with more improvements in 2024!

mesamatrix

Sun, 16 Jun 2019 15:40:01 +0000

You might be familiar with mesamatrix - a nice site to track the state of all GPU drivers provided by Mesa. Ohh.. did I say all? Let’s have a closer look:

This page is a graphical representation of the text file docs/features.txt from the Mesa repository.

So wouldn’t it be sick to get etnaviv mentioned in docs/features.txt and onto the matrix?

First mesamatrix needs to know/support etnaviv. This seems to be quite simple as there is already a pull request to add a bunch of embedded GPU drivers: Add VC4,VC5 and Vivante GPUs

Second we need is to add support for an easy to support extension. After skimming through the list I settled with ARB_seamless_cubemap_per_texture

To be fair I know that there are some UNK bits in the sampler registers and I hope one of them has something to do with seamless cubemaps. I went with a quick try-and-error approach to find the UNK bit which enables seamless cubemaps. And after 20 minutes of hacking the spec@amd_seamless_cubemap_per_texture@amd_seamless_cubemap_per_texture piglit works \o/.

The end result can be found in in this merge request. I hope to merge the changes soon and see something new in mesamatrix.

etnaviv officially landed

Thu, 12 Jan 2017 19:48:36 +0000

After years of hard work the etnaviv team reached an other very important milestone. The gallium driver and the renderonly library got pushed into mesa’s git repository and will be released with mesa 17.0 - yeah!

This does not mean we are done with development at all. There are many interesting topics to work on and some of them like better support for newer IP cores or a reworked GLSL compiler should see the light of day during the next months.

I think 2017 will be a quite interesting one for etnaviv and open source GPU drivers in general.

At this point I want to thank a lot of people who helped out in different areas. The whole mesa community (esp. Rob, Ilia and Emil), SolidRun for provided hardware, Pengutronix - the ‘imx6-company’ - for doing a wonderful job in the kernel space and all the other guys I meet in real life or via irc/mail.

etnaviv: kmscube

Fri, 25 Sep 2015 21:39:10 +0000

Last Friday I got kmscube successfully running with mesa and the new etnaviv DRM kernel driver. At this time I got it not really pixel perfect and I spend some nights to get it fixed. It turns out that I need a small mesa hack to get the rendering correct.

You may ask why did it so long to get it working? Let me explain it to you.

The Vivnate GPU is a so called render-only GPU which does not have any kind of scanout logic in it. It can only render to physical memory and this only in a tiled memory layout. So we need to use the resolve engine found on the GPU to de-tile the rendered image and blit it to the dumb buffer. Currently mesa has no software support for this kind of hardware, but I think that could change soon.

I think that this is a huge step into the right direction. All the needed sources can be found on my github account. At the moment I am cleaning up the code and shortly I will send out RFC patch series for ~~libdrm and~~ mesa. Currently some hacks are needed to get it up an running so there is still a lot left to do.

~~Yes I think libdrm is needed and should not be part of the gallium driver. I even think xf86-video-armada could be using libdrm-etnaviv sooner or later. This one of the topics on my huge TODO list.~~

Russell does not want to maintain a 3rd GPU backend and with that fact I think I will merge the etnaviv libdrm part directly into the driver. Let’s see how the patch series gets accepted.

I think this the perfect time to say “thank you” to some guys. Wladimir thanks for the wonderful code basis you provided for the whole etnaviv project. Rob, thanks for your patience with me – at this point in time I was a complete newbie in the mesa source code and the whole GPU world. Thanks NVIDIA for their wonderful render-only patch. Emil thanks for helping me out with ideas to get render-only into a working state. Also quite important for spare-time open source development is a hardware sponsor. In may case I want to thank SolidRun for sending me two devices for etnaviv development purposes.

Vivante meets devicetree

Sat, 24 May 2014 23:41:14 +0000

It took me some time to rework the device tree bindings but it looks like it starts to work!

Gets loaded into :

[ 3.358155] vivante: module is from the staging directory, the quality is unknown, you have been warned. [ 3.373087] imx-sgtl5000 sound.15: sgtl5000 2028000.ssi mapping ok [ 3.379150] [drm] add child gpu2d [ 3.379154] [drm] add child gpu3d [ 3.380395] vivante-gpu 134000.gpu2d: pre gpu[idx]: 0x00000000 [ 3.380401] vivante-gpu 134000.gpu2d: adding core @idx 1 [ 3.380408] vivante-gpu 134000.gpu2d: post gpu[idx]: 0xecb56c10 [ 3.380460] vivante gpu-subsystem.11: bound 134000.gpu2d (ops gpu_ops [vivante]) [ 3.380467] vivante-gpu 130000.gpu3d: pre gpu[idx]: 0x00000000 [ 3.380473] vivante-gpu 130000.gpu3d: adding core @idx 0 [ 3.380479] vivante-gpu 130000.gpu3d: post gpu[idx]: 0xecb57210 [ 3.380502] vivante gpu-subsystem.11: bound 130000.gpu3d (ops gpu_ops [vivante]) [ 3.380533] IO:R f0200018 14010000 [ 3.380537] IO:R f0200020 00002000 [ 3.380540] IO:R f0200024 00005108 [ 3.380546] vivante gpu-subsystem.11: model: 2000 [ 3.380551] vivante gpu-subsystem.11: revision: 5108 [ 3.380554] IO:R f020001c e0296cad [ 3.380558] IO:R f0200034 c9799eff [ 3.380561] IO:R f0200074 2efbf2d9 [ 3.380564] IO:R f0200084 00000000 [ 3.380567] IO:R f0200088 00000000 [ 3.380572] vivante gpu-subsystem.11: minor_features: c9799eff [ 3.380578] vivante gpu-subsystem.11: minor_features1: 2efbf2d9 [ 3.380583] vivante gpu-subsystem.11: minor_features2: 0 [ 3.380588] vivante gpu-subsystem.11: minor_features3: 0 [ 3.380592] IO:R f0200000 00070100 [ 3.408629] IO:R f0200004 7fffffff [ 3.408632] IO:R f0200000 00070100 [ 3.412940] vivante gpu-subsystem.11: 130000.gpu3d: using IOMMU [ 3.413080] IO:R f01f8018 14010000 [ 3.413083] IO:R f01f8020 00000320 [ 3.413087] IO:R f01f8024 00005007 [ 3.413093] vivante gpu-subsystem.11: model: 320 [ 3.413098] vivante gpu-subsystem.11: revision: 5007 [ 3.413101] IO:R f01f801c e02c7eca [ 3.413105] IO:R f01f8034 c1399eff [ 3.413108] IO:R f01f8074 020fb2db [ 3.413111] IO:R f01f8084 00000000 [ 3.413114] IO:R f01f8088 00000000 [ 3.413120] vivante gpu-subsystem.11: minor_features: c1399eff [ 3.413126] vivante gpu-subsystem.11: minor_features1: 20fb2db [ 3.413131] vivante gpu-subsystem.11: minor_features2: 0 [ 3.413136] vivante gpu-subsystem.11: minor_features3: 0 [ 3.413139] IO:R f01f8000 00070100 [ 3.438615] IO:R f01f8004 7fffffff [ 3.438619] IO:R f01f8000 00070100 [ 3.442583] vivante gpu-subsystem.11: 134000.gpu2d: using IOMMU [ 3.442681] [drm] Initialized vivante 1.0.0 20130625 on minor 1

Vivante MMU v1

Sat, 03 May 2014 20:25:35 +0000

I did spend quite some time the last days to figure out how the MMU v1 could work and what all the code in the v4 Kernel sources does. It took quite some time and a little hint from Russel to finally understand it. So lets start with the technical details.

The MMU uses a page table with a maximum size of 256KB. Where each Page Table Entry (PTE) is 4 byte long. The used page size is 4K and can not be changed via some registers etc. Lets have a look at the bit layout of a PTE.

What does all that code in the v4 driver? In the end it turned out to be a ‘simple’ memory manager. All the code is needed to keep track of which pages are mapped into the GPU address area. Index 0 of the big page table array maps directly to GPU address 0x80000000 and index 1 maps directly to 0x80001000. We can quite easily map Linux kernel pages into the address range of the GPU.

The solution I am working on will use drm_mm as memory manager and the iommu framework to keep the page table in sync.

The next steps for etnaviv

Sat, 22 Feb 2014 17:54:17 +0000

I did spend some time to find the cause for the rendering issues during running some egls2 demos. The fix is a simple one-liner and I would say that GC8xx and GC2000 are ‘equal’ now. That means general work on etnaviv can start now. Currently I am looking in different problem zones and where to start.

Update mesa fork
Start working on an improved compiler
Start working on an kernel interface

I think that Rob will start soon on the kernel interface and the mesa update should be doable during some hours. So I think the next big and important step is to improve the compiler. What should I say… I have basic compiler understanding, did wrote a compiler and VM during my study… thats it. I have never written a single line of glsl and I do not completely understand the Vivante GPU. So a perfect starting point for doing some hacking 🙂 The success rate will be quite low and I hope to not lose my motivation too soon.

Mesa meets GC2000

Sat, 15 Feb 2014 20:18:25 +0000

Here is the first result of running mesatest_gles on mesa&etna:

Following tests are showing good visual results compared to swrast:

Hello_Triangle/CH02_HelloTriangle
Simple_VertexShader/CH08_SimpleVertexShader
CubeVBO/cube_vbo
ParticleSystem/CH13_ParticleSystem
Viewports/viewports

All other are visual corrupted or segfault. So there is still some work done to get all demos up and running.

GC2000 support for etnaviv

Sun, 09 Feb 2014 17:22:56 +0000

Today I hit an important milestone for etnaviv – an open source user-space driver for the Vivante GCxxx series of embedded GPUs. I finally got GC2000 support to a level that it seems to work. It took me some months to get there. At the beginning it sounds easy to rebuild a ‘driver’ if you get readable command buffer dumps. I did start with working on a simple replay program to render a cube in the same was as the binary blob does it. But what should I say… it is quite boring to do everything by hand and not taking advantage of libetnaviv and the ‘driver’ at all. So I decided to go the hard way and try to fix/add all missing bits until it renders something. The good thing is that I have now some knowledge about the structure of libetnaviv, the dirver and mesa in general. It helps a lot if you know why stuff is done that way.

Also this is my first reverse engineering project I contributed to and even I have never done any graphics related stuff – okay I did some OpenGL stuff during my studies.

I try to find some more time to help to create a fully open source graphics stack for Vivante GPUs.

If have not seen it yet: Here is a short video showing the current state of etnaviv on a iMX6q (Sabre Lite). Video in Google+ post