Abstract For many classically chaotic systems it is believed that the quantum wave functions become uniformly distributed, that is the matrix elements of smooth observables tend to the phase space average of the observable. In this paper we study the fluctuations of the matrix elements for the desymmetrized quantum cat map. We present a conjecture for the distribution of the normalized matrix elements, namely that their distribution is that of a certain weighted sum of traces of independent matrices in SU(2). This is in contrast to generic chaotic systems where the distribution is expected to be Gaussian. We compute the second and fourth moment of the normalized matrix elements and obtain agreement with our conjecture.