The study of resonances in open quantum systems with fractal repeller is a fruitful topic in mathematical physics which attained much attention in recent years. The two central questions are the problem of counting resonances which led to the conjecture of the fractal Weyl law and the determination of a spectral gap, i.e. an upper bound on the imaginary part of the quantum resonances.
There are two paradigmatic scattering systems with fractal repeller which provide explicit examples for the study of these questions: The three disk system which is mostly studied in physics and the Laplacian on Schottky surfaces which is the favorite mathematical model.
In the first part of the talk, recent results on the Ruelle resonances of transfer operators of iterated function systems are presented. Furthermore the interest of these results for the study of the resonances on Schottky surfaces is explained.
In the second part the first experimental studies addressing the question of the fractal Weyl law and spectral gap are presented. Using the equivalence between Schrädinger and Helmholtz equation, ``quantum'' n-disk systems have been studied by a microwave setup and the resonances have been extracted by the harmonic inversion. The resulting measured resonance distribution are then compared to the mathematical predictions.