The cast of glee is really singing the songs. They all can sing, although they do pre-record the songs for when they are filming.
Will, the Glee director, in real life played Link Larkin, the male lead in the musical Hairspray, on Broadway. Rachel, in real life become known due to her role in the musical Spring Awakening.
Everyone else is formally trained and the cast has been touring through the summer.
So yeah, the Glee kids do sing.