Note

This blog post is part 1 of a series of blog posts about isaspec and its usage in the etnaviv GPU stack.

I will add here links to the other blog posts, once they are published.

The first time I heard about isaspec, I was blown away by the possibilities it opens. I am really thankful that Igalia made it possible to complete this crucial piece of core infrastructure for the etnaviv GPU stack.

If isaspec is new to you, here is what the Mesa docs have to tell about it:

isaspec provides a mechanism to describe an instruction set in XML, and generate a disassembler and assembler. The intention is to describe the instruction set more formally than hand-coded assembler and disassembler, and better decouple the shader compiler from the underlying instruction encoding to simplify dealing with instruction encoding differences between generations of GPU.

Benefits of a formal ISA description, compared to hand-coded assemblers and disassemblers, include easier detection of new bit combinations that were not seen before in previous generations due to more rigorous description of bits that are expect to be ‘0’ or ‘1’ or ‘x’ (dontcare) and verification that different encodings don’t have conflicting bits (i.e. that the specification cannot result in more than one valid interpretation of any bit pattern).

If you are interested in more details, I highly recommend Rob Clark’s introduction to isaspec presentation.

Target ISA

Vivante uses a fixed-size (128 bits), predictable instruction format with explicit inputs and outputs.

As of today, there are three different encodings seen in the wild:

Base Instruction Set
Extended Instruction Set
Enhanced Vision Instruction Set (EVIS)

Why do I want to switch to isaspec

There are several reasons..

The current state

The current ISA documentation is not very explicit and leaves lot of room for interpretation and speculation. One thing that it provides, are some nice explanations what an instruction does. isaspec does not support <doc> tags yet, but I there is a PoC MR that generates really nice looking and information ISA documentation based on the xml.

I think soon you might find all etnaviv’s isaspec documentation at docs.mesa3d.org.

No unit tests

There are no unit tests based on instructions generated by the blob driver. This might not sound too bad, but it opens the door to generating ‘bad’ encoded instructions that could trigger all sorts of weird and hard-to-debug problems. Such breakages could be caused by some compiler rework, etc.

In an ideal world, there would be a unit test that does the following:

Disassembles the binary representation of an instruction from the blob to a string representation.
Verifies that it matches our expectation.
Assembles the string representation back to 128 bits.
Verifies that it matches the binary representation from the blob driver.

This is our ultimate goal, which we really must reach. etnaviv will not be the only driver that does such deep unit testing - e.g. freedreno does it too.

Easier to understand code

Do you remember the rusticl OpenCL attempt for etnaviv? It contains lines like:

      if (nir_src_is_const(intr->src[1])) {
         inst.tex.swiz = 128;
      }

      if (rmode == nir_rounding_mode_rtz)
         inst.tex.amode = 0x4 + INST_ROUND_MODE_RTZ;
      else /*if (rmode == nir_rounding_mode_rtne)*/
         inst.tex.amode = 0x4 + INST_ROUND_MODE_RTNE;

Do you clearly see what is going on? Why do we need to set tex.amode for an ALU instruction?

I always found it quite disappointing to see such code snippets. Sure, they mimic what the blob driver is doing, but you might lose all the knowledge about why these bits are used that way days after you worked on it. There must be a cleaner, more understandable, and thus more maintainable way to document the ISA better.

This situation might become even worse if we want to support the other encodings and could end up with more of these bad patterns, resulting in a maintenance nightmare.

Oh, and if you wonder what happened to OpenCL and etnaviv - I promise there will be an update later this year.

Python opens the door to generate lot of code

As isaspec is written in Python, it is really easy to extend it and add support for new functionality.

At its core, we can generate a disassembler and an assembler based on isaspec. This alone saves us from writing a lot of code that needs to be kept in sync with all the ISA reverse engineering findings that happen over time.

As isaspec is just an ordinary XML file, you can use any programming language you like to work with it.

One source of truth

I really fell in love with the idea of having one source of truth that models our target ISA, contains written documentation, and extends each opcode with meta information that can be used in the upper layers of the compiler stack.

Missing Features

I think I have sold you the idea quite well, so it must be a matter of some days to switch to it. Sadly no, as there are some missing features:

Only max 64 bits width ISAs are supported
Its home in src/freedreno
Alignment support is missing
No <meta> tags are supported

Add support for 128 bit wide instructions

The first big MR I worked on, extended BITSET APIs with features needed for isaspec. Here we are talking about bitwise AND, OR, and NOT, and left shifts.

The next step was to switch isaspec to use the BITSET API to support wider ISAs. This resulted in a lot of commits, as there was a need for some new APIs to support handling this new feature. After these 31 commits, we were able to start looking into isaspec support for etnaviv.

Decode Support

Now it is time to start writing an isaspec XML for etnaviv, and the easiest opcode to start with is the nop. As the name suggests, it does nothing and has no src’s, no dst, or any other modifier.

As I do not have this initial version anymore, I tried to recreate it - it might have looked something like this:

<?xml version="1.0" encoding="UTF-8"?>
<isa>
<bitset name="#instruction">
	<display>
		{NAME} void, void, void, void
	</display>

	<pattern low="6" high="10">00000</pattern>
	<pattern pos="11">0</pattern>

	<pattern pos="12">0</pattern>
	<pattern low="13" high="26">00000000000000</pattern>
	<pattern low="27" high="31">00000</pattern>

	<pattern pos="32">0</pattern>
	<pattern pos="33">0</pattern>
	<pattern pos="34">0</pattern>
	<pattern low="35" high="38">0000</pattern>
	<pattern pos="39">0</pattern>
	<pattern low="40" high="42">000</pattern>

	<!-- SRC0 -->
	<pattern pos="43">0</pattern> <!-- SRC0_USE -->
	<pattern low="44" high="52">000000000</pattern> <!-- SRC0_REG -->
	<pattern pos="53">0</pattern>
	<pattern low="54" high="61">00000000</pattern> <!-- SRC0_SWIZ -->
	<pattern pos="62">0</pattern> <!-- SRC0_NEG -->
	<pattern pos="63">0</pattern> <!-- SRC0_ABS -->
	<pattern low="64" high="66">000</pattern> <!-- SRC0_AMODE -->
	<pattern low="67" high="69">000</pattern> <!-- SRC0_RGROUP -->

	<!-- SRC1 -->
	<pattern pos="70">0</pattern> <!-- SRC1_USE -->
	<pattern low="71" high="79">000000000</pattern> <!-- SRC1_REG -->
	<pattern low="81" high="88">00000000</pattern> <!-- SRC1_SWIZ -->
	<pattern pos="89">0</pattern> <!-- SRC1_NEG -->
	<pattern pos="90">0</pattern> <!-- SRC1_ABS -->
	<pattern low="91" high="93">000</pattern> <!-- SRC1_AMODE -->
	<pattern pos="94">0</pattern>
	<pattern pos="95">0</pattern>
	<pattern low="96" high="98">000</pattern> <!-- SRC1_RGROUP -->

	<!-- SRC2 -->
	<pattern pos="99">0</pattern> <!-- SRC2_USE -->
	<pattern low="100" high="108">000000000</pattern> <!-- SRC2_REG -->
	<pattern low="110" high="117">00000000</pattern> <!-- SRC2_SWIZ -->
	<pattern pos="118">0</pattern> <!-- SRC2_NEG -->
	<pattern pos="119">0</pattern> <!-- SRC2_ABS -->
	<pattern pos="120">0</pattern>
	<pattern low="121" high="123">000</pattern> <!-- SRC2_AMODE -->
	<pattern low="124" high="126">000</pattern> <!-- SRC2_RGROUP -->
	<pattern pos="127">0</pattern>
</bitset>


<!-- opcocdes sorted by opc number -->

<bitset name="nop" extends="#instruction">
	<pattern low="0" high="5">000000</pattern> <!-- OPC -->
	<pattern pos="80">0</pattern> <!-- OPCODE_BIT6 -->
</bitset></isa>

With the knowledge of the old ISA documentation, I went fishing for instructions. I only used instructions from the binary blob for this process. It is quite important for me to have as many unit tests as I can write to not break any decoding with some isaspec XML changes I do. And it was a huge lifesaver at that time.

After I reached almost feature parity with the old disassembler, I thought it was time to land etnaviv.xml and replace the current handwritten disassembler with a generated one - yeah, so I submitted an MR to make the switch.

As this is only a driver internal disassembler used by maybe 2-3 human beings, it would not be a problem if there were some regressions.

Today I would say the isaspec disassembler is superior to the handwritten one.

Encode Support

The next item on my list was to add encoding support. As you can imagine, there was some work needed upfront to support ISAs that are bigger than 64 bits. This time the MR only contains two commits 😄.

With everything ready it is time to add isaspec based encoding support to etnaviv.

The goal is to drop our custom (and too simple) assembler and switch to one that is powered by isaspec.

This opens the door to:

Modeling special cases for instructions like a branch with no src’s to a new jump instruction.
Doing the NIR src -> instruction src mapping in isaspec.
Supporting different instruction encodings.
Adding meta information to instructions.

Supporting special instructions that are used in compiler unit tests

In the end, all the magic that is needed is shown in the following diff:

diff --git a/src/etnaviv/isa/etnaviv.xml b/src/etnaviv/isa/etnaviv.xml
index eca8241a2238a..c9a3ebe0a40c2 100644
--- a/src/etnaviv/isa/etnaviv.xml
+++ b/src/etnaviv/isa/etnaviv.xml
@@ -125,6 +125,13 @@ SPDX-License-Identifier: MIT
 	<field name="AMODE" low="0" high="2" type="#reg_addressing_mode"/>
 	<field name="REG" low="3" high="9" type="uint"/>
 	<field name="COMPS" low="10" high="13" type="#wrmask"/>
+
+	<encode type="struct etna_inst_dst *">
+		<map name="DST_USE">p->DST_USE</map>
+		<map name="AMODE">src->amode</map>
+		<map name="REG">src->reg</map>
+		<map name="COMPS">p->COMPS</map>
+	</encode>
 </bitset>
 
 <bitset name="#instruction" size="128">
@@ -137,6 +144,46 @@ SPDX-License-Identifier: MIT
 	<derived name="TYPE" type="#type">
 		<expr>{TYPE_BIT2} &lt;&lt; 2 | {TYPE_BIT01}</expr>
 	</derived>
+
+	<encode type="struct etna_inst *" case-prefix="ISA_OPC_">
+		<map name="TYPE_BIT01">src->type &amp; 0x3</map>
+		<map name="TYPE_BIT2">(src->type &amp; 0x4) &gt; 2</map>
+		<map name="LOW_HALF">src->sel_bit0</map>
+		<map name="HIGH_HALF">src->sel_bit1</map>
+		<map name="COND">src->cond</map>
+		<map name="RMODE">src->rounding</map>
+		<map name="SAT">src->sat</map>
+		<map name="DST_USE">src->dst.use</map>
+		<map name="DST">&amp;src->dst</map>
+		<map name="DST_FULL">src->dst_full</map>
+		<map name="COMPS">src->dst.write_mask</map>
+		<map name="SRC0">&amp;src->src[0]</map>
+		<map name="SRC0_USE">src->src[0].use</map>
+		<map name="SRC0_REG">src->src[0].reg</map>
+		<map name="SRC0_RGROUP">src->src[0].rgroup</map>
+		<map name="SRC0_AMODE">src->src[0].amode</map>
+		<map name="SRC1">&amp;src->src[1]</map>
+		<map name="SRC1_USE">src->src[1].use</map>
+		<map name="SRC1_REG">src->src[1].reg</map>
+		<map name="SRC1_RGROUP">src->src[1].rgroup</map>
+		<map name="SRC1_AMODE">src->src[1].amode</map>
+		<map name="SRC2">&amp;src->src[2]</map>
+		<map name="SRC2_USE">rc->src[2].use</map>
+		<map name="SRC2_REG">src->src[2].reg</map>
+		<map name="SRC2_RGROUP">src->src[2].rgroup</map>
+		<map name="SRC2_AMODE">src->src[2].amode</map>
+
+		<map name="TEX_ID">src->tex.id</map>
+		<map name="TEX_SWIZ">src->tex.swiz</map>
+		<map name="TARGET">src->imm</map>
+
+		<!-- sane defaults -->
+		<map name="PMODE">1</map>
+		<map name="SKPHP">0</map>
+		<map name="LOCAL">0</map>
+		<map name="DENORM">0</map>
+		<map name="LEFT_SHIFT">0</map>
+	</encode>
 </bitset>
 
 <bitset name="#src-swizzle" size="8">
@@ -148,6 +195,13 @@ SPDX-License-Identifier: MIT
 	<field name="SWIZ_Y" low="2" high="3" type="#swiz"/>
 	<field name="SWIZ_Z" low="4" high="5" type="#swiz"/>
 	<field name="SWIZ_W" low="6" high="7" type="#swiz"/>
+
+	<encode type="uint8_t">
+		<map name="SWIZ_X">(src &amp; 0x03) &gt;&gt; 0</map>
+		<map name="SWIZ_Y">(src &amp; 0x0c) &gt;&gt; 2</map>
+		<map name="SWIZ_Z">(src &amp; 0x30) &gt;&gt; 4</map>
+		<map name="SWIZ_W">(src &amp; 0xc0) &gt;&gt; 6</map>
+	</encode>
 </bitset>
 
 <enum name="#thread">
@@ -272,6 +326,13 @@ SPDX-License-Identifier: MIT
 			</expr>
 		</derived>
 	</override>
+
+	<encode type="struct etna_inst_src *">
+		<map name="SRC_SWIZ">src->swiz</map>
+		<map name="SRC_NEG">src->neg</map>
+		<map name="SRC_ABS">src->abs</map>
+		<map name="SRC_RGROUP">p->SRC_RGROUP</map>
+	</encode>
 </bitset>
 
 <bitset name="#instruction-alu-no-src" extends="#instruction-alu">

One nice side effect of this work is the removal of isa.xml.h file that has been part of etnaviv since day one. We are able to generate all the file contents with isaspec and some custom python3 scripts. The move of instruction src swizzling from the driver into etnaviv.xml was super easy - less code to maintain!

Summary

I am really happy with the end result, even though it took quite some time from the initial idea to the point when everything was integrated into Mesa’s main git branch.

There is so much more to share - I can’t wait to publish parts II and III.

Target ISA#

Why do I want to switch to isaspec#

The current state#

No unit tests#

Easier to understand code#

Python opens the door to generate lot of code#

One source of truth#

Missing Features#

Add support for 128 bit wide instructions#

Decode Support#

Encode Support#

Supporting special instructions that are used in compiler unit tests#

Summary#