Skip to content

Add ACLE intrinsics for unprivileged pair loads/stores #472

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
CarolineConcatto wants to merge 1 commit into
base: main
Choose a base branch
from
+97 −0
Open

Add ACLE intrinsics for unprivileged pair loads/stores #472

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions neon_intrinsics/advsimd.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ See [Changelog for year 2026](#changelog-for-year-2026).

### Changelog for year 2026

* Added proposed support for FEAT_LSUI LDTP/STTP intrinsics.
* Added support for FEAT_F16F32DOT
* Added support for FEAT_F16F32MM and FEAT_F16MM
* Renamed vmmlaq_f16_f16 to vmmlaq_f16
Expand Down Expand Up @@ -3829,6 +3830,30 @@ The intrinsics in this section are guarded by the macro ``__ARM_NEON``.

### Load



#### Unprivileged pair

The proposed FEAT_LSUI AArch64 intrinsics expose LDTP and STTP for two 128-bit SIMD&FP registers.

| Intrinsic | Argument preparation | AArch64 Instruction | Result | Supported architectures |
|---|---|---|---|---|
| `int8x16x2_t vldtq_s8_x2(int8_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `int16x8x2_t vldtq_s16_x2(int16_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `int32x4x2_t vldtq_s32_x2(int32_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `int64x2x2_t vldtq_s64_x2(int64_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `uint8x16x2_t vldtq_u8_x2(uint8_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `uint16x8x2_t vldtq_u16_x2(uint16_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `uint32x4x2_t vldtq_u32_x2(uint32_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `uint64x2x2_t vldtq_u64_x2(uint64_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `float16x8x2_t vldtq_f16_x2(float16_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `bfloat16x8x2_t vldtq_bf16_x2(bfloat16_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `float32x4x2_t vldtq_f32_x2(float32_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `float64x2x2_t vldtq_f64_x2(float64_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |
| `mfloat8x16x2_t vldtq_mf8_x2(mfloat8_t const *ptr)` | `ptr -> Xn` | `LDTP Qt1,Qt2,[Xn]` | `Qt1/Qt2 -> result.val[0]/result.val[1]` | `A64` |

The architectural signed immediate is not exposed by this pointer-only API; normal compiler address formation may be used when an offset is needed.

#### Stride

| Intrinsic | Argument preparation | AArch64 Instruction | Result | Supported architectures |
Expand Down Expand Up @@ -4308,6 +4333,24 @@ The intrinsics in this section are guarded by the macro ``__ARM_NEON``.

### Store

#### Unprivileged pair

| Intrinsic | Argument preparation | AArch64 Instruction | Result | Supported architectures |
|---|---|---|---|---|
| `void vsttq_s8_x2(int8_t *ptr, int8x16x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_s16_x2(int16_t *ptr, int16x8x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_s32_x2(int32_t *ptr, int32x4x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_s64_x2(int64_t *ptr, int64x2x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_u8_x2(uint8_t *ptr, uint8x16x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_u16_x2(uint16_t *ptr, uint16x8x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_u32_x2(uint32_t *ptr, uint32x4x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_u64_x2(uint64_t *ptr, uint64x2x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_f16_x2(float16_t *ptr, float16x8x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_bf16_x2(bfloat16_t *ptr, bfloat16x8x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_f32_x2(float32_t *ptr, float32x4x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_f64_x2(float64_t *ptr, float64x2x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |
| `void vsttq_mf8_x2(mfloat8_t *ptr, mfloat8x16x2_t val)` | `val.val[0]/val.val[1] -> Qt1/Qt2`<br>`ptr -> Xn` | `STTP Qt1,Qt2,[Xn]` | | `A64` |

#### Stride

| Intrinsic | Argument preparation | AArch64 Instruction | Result | Supported architectures |
Expand Down
1 change: 1 addition & 0 deletions neon_intrinsics/advsimd.template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ See [Changelog for year 2026](#changelog-for-year-2026).

### Changelog for year 2026

* Added proposed support for FEAT_LSUI LDTP/STTP intrinsics.
* Added support for FEAT_F16F32DOT
* Added support for FEAT_F16F32MM and FEAT_F16MM
* Renamed vmmlaq_f16_f16 to vmmlaq_f16
Expand Down
27 changes: 27 additions & 0 deletions tools/intrinsic_db/advsimd.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4863,3 +4863,30 @@ float32x4_t vmmlaq_f32_f16(float32x4_t r, float16x8_t a, float16x8_t b) r -> Vd.

<SECTION> Non-widening half-precision matrix multiply instruction. Requires the +f16mm architecture extension.
float16x8_t vmmlaq_f16(float16x8_t r, float16x8_t a, float16x8_t b) r -> Vd.8H;a -> Vn.8H;b -> Vm.8H FMMLA Vd.8H, Vn.8H, Vm.8H Vd.8H -> result A64
<SECTION> Unprivileged pair with non-temporal hint
int8x16x2_t vldtq_s8_x2(int8_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
int16x8x2_t vldtq_s16_x2(int16_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
int32x4x2_t vldtq_s32_x2(int32_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
int64x2x2_t vldtq_s64_x2(int64_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
uint8x16x2_t vldtq_u8_x2(uint8_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
uint16x8x2_t vldtq_u16_x2(uint16_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
uint32x4x2_t vldtq_u32_x2(uint32_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
uint64x2x2_t vldtq_u64_x2(uint64_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
float16x8x2_t vldtq_f16_x2(float16_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
bfloat16x8x2_t vldtq_bf16_x2(bfloat16_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
float32x4x2_t vldtq_f32_x2(float32_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
float64x2x2_t vldtq_f64_x2(float64_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
mfloat8x16x2_t vldtq_mf8_x2(mfloat8_t const *ptr) ptr -> Xn LDTP Qt1,Qt2,[Xn] Qt2 -> result.val[1];Qt1 -> result.val[0] A64
void vsttq_s8_x2(int8_t *ptr, int8x16x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_s16_x2(int16_t *ptr, int16x8x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_s32_x2(int32_t *ptr, int32x4x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_s64_x2(int64_t *ptr, int64x2x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_u8_x2(uint8_t *ptr, uint8x16x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_u16_x2(uint16_t *ptr, uint16x8x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_u32_x2(uint32_t *ptr, uint32x4x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_u64_x2(uint64_t *ptr, uint64x2x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_f16_x2(float16_t *ptr, float16x8x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_bf16_x2(bfloat16_t *ptr, bfloat16x8x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_f32_x2(float32_t *ptr, float32x4x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_f64_x2(float64_t *ptr, float64x2x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
void vsttq_mf8_x2(mfloat8_t *ptr, mfloat8x16x2_t val) val.val[1] -> Qt2;val.val[0] -> Qt1;ptr -> Xn STTP Qt1,Qt2,[Xn] A64
26 changes: 26 additions & 0 deletions tools/intrinsic_db/advsimd_classification.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4745,3 +4745,29 @@ vdot_laneq_f32_f16 Vector arithmetic|Dot product
vdotq_lane_f32_f16 Vector arithmetic|Dot product
vmmlaq_f32_f16 Vector arithmetic|Matrix multiply
vmmlaq_f16 Vector arithmetic|Matrix multiply
vldtq_s8_x2 Load|Stride
vldtq_s16_x2 Load|Stride
vldtq_s32_x2 Load|Stride
vldtq_s64_x2 Load|Stride
vldtq_u8_x2 Load|Stride
vldtq_u16_x2 Load|Stride
vldtq_u32_x2 Load|Stride
vldtq_u64_x2 Load|Stride
vldtq_f16_x2 Load|Stride
vldtq_bf16_x2 Load|Stride
vldtq_f32_x2 Load|Stride
vldtq_f64_x2 Load|Stride
vldtq_mf8_x2 Load|Stride
vsttq_s8_x2 Store|Stride
vsttq_s16_x2 Store|Stride
vsttq_s32_x2 Store|Stride
vsttq_s64_x2 Store|Stride
vsttq_u8_x2 Store|Stride
vsttq_u16_x2 Store|Stride
vsttq_u32_x2 Store|Stride
vsttq_u64_x2 Store|Stride
vsttq_f16_x2 Store|Stride
vsttq_bf16_x2 Store|Stride
vsttq_f32_x2 Store|Stride
vsttq_f64_x2 Store|Stride
vsttq_mf8_x2 Store|Stride
Loading