Abstract: A local normal form theorem for smooth equivariant maps between Fr\'echet manifolds is established. Moreover, an elliptic version of this theorem is obtained. The proof these normal form results is inspired by the Lyapunov-Schmidt reduction for dynamical systems and by the Kuranishi method for moduli spaces, and uses a slice theorem for Fr\'echet manifolds as the main technical tool. As a consequence of this equivariant normal form theorem, the abstract moduli space obtained by factorizing a level set of the equivariant map with respect to the group action carries the structure of a Kuranishi space. Moreover, the theory of singular symplectic reduction is developed in the infinite-dimensional Fr\'echet setting. By refining the above construction, a normal form for momentum maps similar to the classical Marle-Guillemin-Sternberg normal form is established. Analogous to the reasoning in finite dimensions, this normal form result is then used to show that the reduced phase space decomposes into smooth manifolds each carrying a natural symplectic structure. Finally, the singular symplectic reduction scheme is further investigated in the situation where the original phase space is an infinite-dimensional cotangent bundle. The fibered structure of the cotangent bundle yields a refinement of the usual orbit-momentum type strata into so-called seams. Using a suitable normal form theorem, it is shown that these seams are manifolds. Taking the harmonic oscillator as an example, the influence of the seams on dynamics is illustrated. The general results stated above are applied to various gauge theory models. The moduli spaces of anti-self-dual connections in four dimensions and of Yang-Mills connections in two dimensions is studied. Moreover, the stratified structure of the reduced phase space of the Yang-Mills-Higgs theory is investigated in a Hamiltonian formulation.