Background

Sparse matrices (matrices with zeroes) are very common in both scientific and machine learning applications. By skipping zeroes, otherwise wasteful computations are avoided. There have been prior studies concerning how to store such sparse matrices as well as hardware techniques to compute with them. This technology is based on the key insight that in several machine learning applications, the sparsity of matrices is low to moderate. At such sparsity, it is often inefficient to compute over sparse data as the cost of metadata can outweigh the savings in storage.

Technology Overview

This technology is a new component of computer chips. By “computer chips”, we refer to chips that are used in a variety of computing platforms including servers, desktops, mobiles, automobiles, sensors, etc. The component is a piece of hardware that is added between the processing element of the chip and the memory system. The hardware provides the following functions:

It reads compressed metadata from memory and supplies expanded data to the processing element.
It receives expanded data from the processing element and stores compressed metadata into memory.
It reads compressed metadata from memory and uses the non-zero location information to automatically fetch data from vectors at the computed indices.

Stage of Development

Prototype of the proposed hardware has been developed as a software simulation model.
Experimental evaluation of the performance and energy benefits of the proposed hardware have been partially investigated.

Benefits

Automatically expanding the data simplifies the programming model allowing software to become independent of the metadata format and resulting overhead. The software running on the processing element is unaware of the format and does not incur this overhead in performance or energy. This architecture enables better processing-memory overlap: while the processing element is working on one batch of matrix data, the proposed hardware prefetches and prepares data required for the next batch. This overlap also results in improving performance. The hardware is designed to be either configurable or fully programmable. In low-end chips such as those used in sensors, the hardware has to achieve very low area and power and therefore it is a configurable block with several pre-set options. In higher-end systems, the hardware is fully programmable.

Applications

Companies who manufacture or implement chips in a way that would benefit from this technology.

Simultaneously Improving Performance and Energy of Sparse Matrix Computations at Low Sparsity

IP Status

Seeking