CellPerformance

Fast Matrix Multiplication on Cell (SMP) Systems

Tue, 10 Jul 2007 07:17:23 +0000

Daniel Hackenberg wrote to tell me about some matrix multiply code he has written for the Cell.

From his page:

This site describes a fast matrix multiplication code for Cell BE processors. It has been developed as part of a seminar paper at the Center for Information Services and High Performance Computing. The program is freely available under the GNU GPL.

Go ahead and check it out: Fast Matrix Multiplication on Cell (SMP) Systems [tu-desden.de]

Cleaning House

Fri, 29 Jun 2007 07:43:27 +0000

UPDATE! 7 July 2007 The new CellPerformance Forums are now up and running, hosted by our friends at Beyond3D. [Thanks guys!]

I'll be fixing up the links and generally cleaning things up to point all article discussions over to the new forums. It might take a little time, so be patient - but the quality of their forums is great, and I know that the addition of the existing B3D community to our own will drive a lot of good discussion.

Remember the main articles will continue to be posted here. Hopefully, a few more than I've had time for in recent months.

Well be back up and running full-speed shortly!

Mike.

Hey everyone! I know our forums have been hacked. You'd think that these kids would have better things to do. You'd also think that they'd appreciate exactly the kind of info we're trying to share here. Dumb.

Anyway, not worth the effort to worry about them. I'm working on a plan that will make the forums better and more useful. And hopefully, I can get a little help from some friends.

Stay tuned. It's time for me to get back to this and get all of you more of the info you want!

Mike.

Handy PS3 Linux Framebuffer Utilities

Sat, 31 Mar 2007 06:14:13 +0000

While the documentation within Sony's vsync example should be enough to get you started with writing to the framebuffer, here's a couple of handy functions to test the framebuffer settings, open the virtual terminal and get access the the frame buffer.

Open the virtual terminal:
cp_vt.h
cp_vt.c

Open the framebuffer:
cp_fb.h
cp_fb.c

Dump framebuffer info:
fb_info.c

Example output from fb_info
Example of using cp_vt and cp_fb

Files should be compiled with:

ppu-gcc -std=c99 -pedantic -W -Wall -O3

HowTo: Huge TLB pages on PS3 Linux

Tue, 30 Jan 2007 04:26:49 +0000

Updated! (22 Mar 07) Minor edits. Added notes for YellowDog Linux. Added source code for using huge page allocation.
Updated! (30 Mar 07) A couple minor fixes. Thanks to Guénaël Renault for pointing them out!
Updated! (15 July 07) Added notes for kernel 2.6.21

Guest article: Understanding the TLB and minimizing misses is a critical part of high performance Cell programming. Unfortunately some PS3 kernels do not come with huge page support enabled. Jakub Kurzak and Alfredo Buttari step through the details of recompiling the kernel for huge page support.

The availability of huge TLB pages depends on the way the linux kernel has been configured prior to compilation. The default kernel that ships with Fedora Core 5 (most likely with any other distribution that has binary kernel packages) doesn't include this option. So, in order to have huge TLB pages, it is necessary to reconfigure the kernel, recompile it, instruct the boot loader about the newly created kernel image. Finally we will also show a way to allocate the TLB pages automatically at boot time.

[Mike Acton] This process also works with YellowDog Linux virtually unchanged.

Cross-compiling for PS3 Linux

Wed, 29 Nov 2006 08:13:56 +0000

Now that the PS3 is out and multiple Linux-based distributions are available which can be installed using Open Platform [playstation.com] it's time to start developing on some publically available hardware!

Although the PPU and SPU compilers can be installed and used on the PS3 directly, I find it much more familiar and convinient to cross-compile from my desktop and just ship the resulting executables over to the target (PS3).

In this article, I will detail the basic steps I used to get started building on a host PC and running on the PS3.

Unaligned scalar load and store on the SPU

Fri, 15 Sep 2006 10:59:18 +0000

Albert Noll, a student at UC Irvine is working on an interesting project. According to him:

"I am currently working on a java virtual machine runtime environment which hides the heterogenity of the cell architecture. Conventional java code code be executed and benefit from the numerous execution units the Cell architecture offers. I am doing some java benchmarks (java grande) to test the efficiancy of the implementation, but I still have some problems achieving really good results."

One of the problems Albert has encountered recently is in loading and storing scalar doubles to the java stack. He recently posed a question on the Cell Broadband Engine Architecture forum [ibm.com]:

"I have the following problem: I want to load a double value from an array (represents stack of the application) which is of type unsigned int. The two 32-bit values, which represent the double value have been casted to a double before, so the bits are set according to the double representation of the value."

The solution to this problem is to remember that the SPU does not have a scalar instruction set or access local memory in anything except 16 bytes quadwords. The ability to compile scalar code on the SPU is something of a convinience, but it doesn't come without a penalty.

The first step, before considering performance, is to properly be able to load and store the unaligned double values.

atan2 on SPU

Tue, 12 Sep 2006 10:21:52 +0000

On 2006 March 03 on the IBM developerWorks Cell Broadband Engine Architecture forum [ibm.com] an interesting question was asked:

"I am trying to port an application from an older version of SDK to SDK 1.0. It uses atan2(.....) function, which is causing trouble... This code worked fine on SDK28, but now it looks like the new functions dont have this particular function defined..
I did change the makefile to include $(SDKLIB)/libmath.a

I searched in ./sysroot/usr/spu/include/* and src/include/spu/* but couldn't find a headerfile that has it defined.

Can anyone please suggest if I should just change the code to not use that function or is there a way to invoke it still?

Thanks!"

It turned out this function was not available in the SDK.

The following is a branch-free implementation of atan2 vector floats for the SPU. A scalar version which simply casts to vector and back is also provided. This implementation is fairly quick-and-dirty and no particular level of accuracy is gauranteed, but it should be usable for many purposes.

static inline vector float cp_fatan( const vector float x );
static inline float cp_fatan_scalar( const float x );
static inline vector float cp_fatan2( const vector float y, const vector float x );
static inline float cp_fatan2_scalar( const float y, const float x );

Or download the source files:
cp_fatan-cbe-spu.h
cp_fatan-cbe-spu.c

This code is C99 source. For gcc, use the following flags: -std=c99 -pedantic

Branch-free implementation of half-precision (16 bit) floating point

Mon, 17 Jul 2006 09:20:36 +0000

Update! (19 July 06) Added Multiply. Fixed a problem with using __builtin_clz().

Update! (17 July 06) The code has been considerably refactored. Decided to go with single function per expression. The expressions have been reduced as a first optimization pass.

Project

The goal of this project is serve as an example of developing some relatively complex operations completely without branches - a software implementation of half-precision floating point numbers (That does not use floating point hardware). This example should echo the IEEE 754 standard for floating point numbers as closely as reasonable, including support for +/- INF, QNan, SNan, and denormalized numbers. However, exceptions will not be implemented.

Half-precision floats are used in cases where neither the range nor the precision of 32 bit floating point numbers are needed, but where some dynamic precision is required. Two common uses are for image transformation, where the range of each component (e.g. red, green, blue, alpha) is typically limited to or near [0.0,1.0] or vertex data (e.g. position, texture coordinates, color values, etc.).

The main advantage of half-precision floats is their size. Beyond the considerable potential for memory savings, processing a large number of half-precision values is more cache-friendly than using 32 bit values.

The current released version (including tests) can be downloaded here: half.tgz

half_to_float() Convert Half To Float (Scalar Version)
half_from_float() Convert Float to Half (Scalar Version)
half_add() Half Add (Scalar Version)
half_sub() Half Subtract (Scalar Version)
half_mul() Half Multiply (Scalar Version)

Better Performance Through Branch Elimination

Tue, 11 Jul 2006 04:30:07 +0000

Update! (11 July 06) Major Revision. With much help from André de Leiradella, these are improved working drafts of the series on branch elimination. Now included are a more detailed background on branches and many more examples!

Introduction

Second only to poor data access patterns, branches can have a big negative impact in the performance of a program. Methods for reducing branch penalties, such as both dynamic and static (software-assisted) branch prediction hardware, despite their successes, are increasingly less effective as the length of the instruction pipelines increase, particularly with in-order architectures where execution must be stalled when hardware prediction fails.

Branching, both conditional and unconditional, slows most implementations. Even an unconditional branch or a correctly predicted taken branch may cause a delay if the target instruction is not in the fetch buffer or the cache. It is therefore best to use branch instructions carefully and to devise algorithms that reduce branching. Many operations that normally use branches may be performed either with fewer or no branches.

-- From IBM's The PowerPC Compiler Writer's Guide 3.1.5

Branches represent a significant part of both performance critical and general purpose code - as a general rule of thumb, 20% of the instructions in typical code are branches. In inner loops and other code sections which demand the highest performance may benefit from a multifold increase in performance by eliminating, or reducing, branches.

This series of articles will present the types of delays that branches may cause in program execution and some programming patterns that help avoid those delays.

Part 1: Introduction
Part 2: Background on Branching
Part 3: Benefits to Branch Elimination
Part 4: Programming with Branches, Patterns and Tips
Part 5: More Techniques for Eliminating Branches

Additional Examples:

Increment And Decrement Wrapping Values
Occasionally you have a set of values that you want to wrap around as you increment and decrement them. But the straightfoward implementation can have a big impact on processors where comparisons and branches are expensive (e.g. PowerPC). This article presents a straightforward branch-free implementation of these functions.

Choosing to Avoid Branches: A Small Altivec Example
An example of why less instructions doesn't always equal faster code.

Branch-free implementation of half-precision (16 bit) floating point
The goal of this project is serve as an example of developing some relatively complex operations completely without branches - a software implementation of half-precision floating point numbers.

If this article interests you, I recommend highly Henry S. Warren's book Hacker's Delight and his associated website, Hacker's Delight. This book is a must-have for every programmer.

Box Overlap

Sun, 18 Jun 2006 11:21:21 +0000

Background

Interactive 3D applications frequently need to check whether one geometric object overlaps another. In this article, we'll look at a function to test for overlap between 3D boxes, and we'll show how to optimize this function for the CBE.

A 4x4 Matrix Inverse

Sat, 03 Jun 2006 09:17:03 +0000

GUEST ARTICLE! Cédric Lallain is a Frenchman who has been working with me on Cell/PS3 research at Highmoon Studios in Carlsbad, CA.. I hope that this is only the first of many contributions to the community by Cédric. Welcome aboard! -- Mike.

Inverse matrix on PPU and on SPU using SIMD instructions.

This article will talk about how to convert some scalar code to SIMD code for the PPU and SPU using the inverse matrix as an example.

Most of the time in the video games, programmers are not doing a standard inverse matrix. It is too expensive. Instead, to inverse a matrix, they consider it as orthonormal and they just do a 3x3 transpose of the rotation part with a dot product for the translation. Sometimes the full inverse algorithm is necessary.

The main goal is to be able to do it as fast as possible. This is why the code should use SIMD instructions as much as possible.

A vector is an instruction operand containing a set of data elements packed into a one-dimensional array. The elements can be fixed-point or floating-point values. Most Vector/SIMD Multimedia Extension and SPU instructions operate on vector operands. Vectors are also called Single-Instruction, Multiple-Data (SIMD) operands, or packed operands.
SIMD processing exploits data-level parallelism. Data-level parallelism means that the operations required to transform a set of vector elements can be performed on all elements of the vector at the same time. That is, a single instruction can be applied to multiple data elements in parallel.

[Chapter 2.5.1 in the released pdf by IBM: Cell Broadband Engine Programming Handbook [ibm.com]].

Each SPE is a 128-bit RISC processor specialized for data-rich, compute-intensive SIMD and scalar applications.

[Chapter 3 in the released pdf by IBM: Cell Broadband Engine Programming Handbook [ibm.com]].

Also the number of branches should stay to the strict minimum. Any extra branches will slow down the final solution. For more information about it, check the article: Better Performance Through Branch Elimination [CellPerformance.com]. The first step is to choose the most suitable algorithm in order to reach the objectives. Different algorithms exist to inverse a matrix:

The Gauss-Jordan elimination: The Gauss-Jordan elimination is a method to find the inverse matrix solving a system of linear equations. A good explanation about how this algorithm work can be found in the book "Numerical Recipes in C" [library.cornell.edu] chapter 2.1.
For a visual demonstration using a java applet see: Gauss-Jordan Elimination [cse.uiuc.edu]. In this algorithm, the choice of a good pivot is a critical part. To do it, all floating point values of a specific column need to be tested with each other, one by one. This, by definition, doesn't suit very well in SIMD code.
Performing the algorithm, some multiplications are be done between columns (e.g.: to apply the pivot) and some other operations between rows (e.g.: to apply the multiplier to the rest of the matrix). This requires extra code to swap rows and columns in order to use SIMD instructions.

Inversion using LU decomposition: The description of the inverse calculation can be found in "Numerical Recipes in C" [library.cornell.edu] chapter 2.3.

In linear algebra, a block LU decomposition is a decomposition of a block matrix into a lower block triangular matrix L and an upper block triangular matrix U. This decomposition is used in numerical analysis to reduce the complexity of the block matrix formula.

[Block LU decomposition [wikipedia.org]]

This algorithm would probably be very useful if the size of the matrix was 8x8. In this case, it requires doing the calculation two floating points at a time where a vector type contains four.

Inversion by Partitioning: To inverse a matrix A (size N) by partitioning, the matrix is partitioned into:

       |  A0    A1  |
   A = |            | with A0 and A3 squared matrix with the respective size
       |  A2    A3  |                s0 and s3 following the rule: s0 + s3 = N

The inverse is

          |  B0    B1  |
   InvA = |            |
          |  B2    B3  |

with:

  B0 = Inv(A0 - A1 * InvA3 * A2)
  B1 = - B0 * (A1 * InvA3)
  B2 = - (InvA3 * A2) * B0
  B3 = InvA3 + B2 * (A1 * InvA3)

More information can be found in "Numerical Recipes in C" [library.cornell.edu] chapter 2.7

The issue related above is also present here; the idea is to work four floating points at a time and not only two.

Using the inverse formula ( (1/det(M)) * Transpose(Cofactor(M))): Check the article about Matrix Inverse [mathworld.wolfram.com] for more information about this formula.

This is the algorithm which will be used to inverse the matrix. Each step presents a very good factorization ratio; it's possible to group the operations in order to replace them by SIMD instructions.
The most critical part in this algorithm is the calculation of all cofactors. This part has also two great advantages for our objectives. It's 100% calculation; this allows writing code without branching. All cofactor values are computed the same way and can be computed in parallel and independently of each other. This is a perfect place to use the SIMD instructions.

This article will start with a basic implementation of the inverse formula using scalar instructions. Then this code will be modified to prepare the SIMD version. The first SIMD version will be done for the PPU. The final one will be conversion using the SPU intrinsic instruction set.

Avoiding Microcoded Instructions On The PPU

Fri, 28 Apr 2006 04:45:14 +0000

What are microcoded instructions?

Microcode is a special instruction set that is (usually) only available to the hardware. On the PPU (PowerPC Unit), small microprograms made up of microcode are stored in ROM and executed in the place of those PowerPC instructions that were too costly to implement directly in hardware or do not fit into the pipeline design very well. The size of a microprogram is measured in microwords.

The PowerPC instructions for which a microprogram is executed are often called microcoded instructions.

Microcoded instructions may be conditionally executed or unconditionally executed. Unconditionally executed microcoded instructions always execute the microprogram. Conditionally executed microcoded instructions will only execute the microprogram when the values of the register operands are exceptional in some way. Microcoded instructions are a special case of normal instructions and conditionally executed microcoded instructions are a special case of those.

Why avoid microcoded instructions?

The G5 core implements several instructions in microcode. These instructions cause a pipeline bubble during decode. The most commonly used microcoded instructions are load and store multiple -- lmw and stmw. These are often generated by the compiler to save space when saving and restoring registers on the stack. You can force GCC to avoid these instructions by specifying -mnomultiple. Indexed forms and/or algebraic forms of updating load and stores are also executed as microcode. You can force GCC to avoid these instructions by specifying -mno-update.

-- From G5 Performance Primer [apple.com]

Like the G5, the PPU contains microcoded instructions. Microcoded instructions are implemented in order to maintain compatibility with the PowerPC standard (a processor can only be called a PowerPC processor if it adheres to the standard [ibm.com].) When one of these instructions is decoded, the current pipeline is flushed, the microded program is then fetched from ROM and executed as a single atomic unit. The process of flushing the pipeline, fetching the microcode and executing the program takes quite a long time compared to other instructions. Additionally, because the instruction must be executed atomically in order to remain as transparent to the user as possible, any resources needed by the microcode program must be locked.

;; micr insns will stall at least 7 cycles to get the first instr from ROM, micro instructions are not dual issued.

-- From cellpu.md (CBE Toolchain 2.3 source code [bsc.es])

The minimum seven (7) cycle stall for microcoded instructions is derived from the fixed stages of the microcode section of the instruction pipeline. Microcoded stages are inserted after the last instruction buffer stage and before the first instruction decode stage. The actual penalty is determined by the complexity and length of the instruction.

For more information on the PPU pipeline stages see: Introduction to the Cell multiprocessor [ibm.com]

The details on which instructions are microcoded and the associated penalties are specific to each PowerPC device and are outlined in the User's Guide for the individual processor. For example, see the IBM PowerPC 970FX RISC Microprocessor User's Guide [ibm.com] paying particular attention to Section 6.3.3 Instruction Decode, Cracking, and Microcode.

The PPU User's Guide has not been released publically. So how is a programmer to know which instructions are microcoded and how to avoid them?

Read on to find out.

UPDATE: 11 MAY 2006

On May 10, 2006 IBM released the Cell Broadband Engine Programming Handbook [ibm.com]. Section A.1.3.1 (Unconditionally Microcoded Instructions) has a detailed list of those instructions which are always microcoded, including latency information and microword count. Before this document was released there were no public documents which described in detail, the penalties for using microcoded instructions. This article has been updated to reflect those details.

From the document:

Note: A minimum of 11 cycles are required before the first instruction is received from the microcode ROM, so microcoded instructions should be avoided if possible.

Most microcoded instructions are decoded into two or three simple PowerPC instructions, and they can be avoided in most cases. The microcoded instructions are typically decomposed into an integer and a load or store operation, with a dependency between them. Although most microcoded PowerPC instructions are decoded into only a few simple instructions, it is important to keep in mind that there are typically dependencies between the internal operations of the microcode, which generate stalls at the issue stage. Replacing the microcoded instructions with PowerPC instructions not only avoids stalling but also gives more latitude in scheduling instructions to avoid stalls, as well as potentially improving multithreaded performance.

Choosing to Avoid Branches: A Small Altivec Example

Thu, 20 Apr 2006 07:54:25 +0000

Balancing speed and instruction count

I came across an interesting bit of Altivec code [simdtech.org] on the Altivec mailing list [simdtech.org] today and I thought it would make a good example of how speed is not simply a matter of instruction count on the PPU and SPU.

The function (modified slightly for this example):

vector unsigned short test( vector unsigned short a, vector unsigned short b ) { const vector unsigned short mask = (vector unsigned short)vec_cmplt( a, b ); if (vec_all_ge(a, b)) { return (a); } return (mask); }

Ingoring the function stack push/pop, we would have a very few instructions. Something like:

vcmpgtuh. v0, v3, v2 mfcr r0 rlwinm r0, r0, 27, 1 vcmpgtuh v0, v3, v2 cmpwi cr7, r0, 0 bne+ cr7, L2 vor v2, v0, v0 L2:

Ben Weiss [shellandslate.com] wrote this snippet to demonstrate that GCC does not optimize out redundant calls when Altivec instructions are called both with and without the CR-modify bit set. It's worth noting that this code (in the general case) will be a performance loss. Not only will there very likely be performance issues in these instructions themselves, but it will also affect the surrounding code by creating an optimization barrier.

Read on to find out more about the issues and examine the alternative...

More Techniques for Eliminating Branches

Tue, 11 Apr 2006 08:00:21 +0000

Better Performance Through Branch Elimination

Part 1: Introduction
Part 2: Background on Branching
Part 3: Benefits to Branch Elimination
Part 4: Programming with Branches, Patterns and Tips
Part 5: More Techniques for Eliminating Branches

Additional Examples:

Increment And Decrement Wrapping Values
Occasionally you have a set of values that you want to wrap around as you increment and decrement them. But the straightfoward implementation can have a big impact on processors where comparisons and branches are expensive (e.g. PowerPC). This article presents a straightforward branch-free implementation of these functions.

Choosing to Avoid Branches: A Small Altivec Example
An example of why less instructions doesn't always equal faster code.

Branch-free implementation of half-precision (16 bit) floating point
The goal of this project is serve as an example of developing some relatively complex operations completely without branches - a software implementation of half-precision floating point numbers.

Programming with Branches, Patterns and Tips

Tue, 11 Apr 2006 07:35:33 +0000

Better Performance Through Branch Elimination

Part 1: Introduction
Part 2: Background on Branching
Part 3: Benefits to Branch Elimination
Part 4: Programming with Branches, Patterns and Tips
Part 5: More Techniques for Eliminating Branches

Additional Examples:

Increment And Decrement Wrapping Values
Occasionally you have a set of values that you want to wrap around as you increment and decrement them. But the straightfoward implementation can have a big impact on processors where comparisons and branches are expensive (e.g. PowerPC). This article presents a straightforward branch-free implementation of these functions.

Choosing to Avoid Branches: A Small Altivec Example
An example of why less instructions doesn't always equal faster code.

Branch-free implementation of half-precision (16 bit) floating point
The goal of this project is serve as an example of developing some relatively complex operations completely without branches - a software implementation of half-precision floating point numbers.