Thoma learned to paint from a clock-face maker. He was born in Bernau, in the Black Forest, in 1839; his father was a woodworker, his mother came from a family of artisans. He started as a lithographer's apprentice in Basel, then as a clock-face painter in Furtwangen, completing neither apprenticeship before entering the Karlsruhe Academy in 1859.

A trip to Paris in 1868 with his friend Otto Scholderer exposed him to Courbet and the Barbizon painters, whose realism influenced his landscape style. He moved to Munich and spent six years there, then to Frankfurt, where he lived from 1876 to 1899. He also spent extended periods in Italy, becoming one of the "German Romans", artists who found in Renaissance observation a means of contemporary expression that fed into European Symbolism.

His landscapes of the Black Forest, with their deep greens, rounded hills and pastoral stillness, made him the best-known painter of that region. He also painted mythological and Symbolist subjects, self-portraits with allegorical figures, and genre scenes of German rural life. He married his student Cella Berteneder, who became known as a painter of flowers and still lifes.

In 1899 he was appointed director of the Kunsthalle Karlsruhe, a position he held until 1919. After his death in 1924, his work was appropriated by nationalist and Nazi ideology, and several paintings were looted from Jewish collectors during the Third Reich. The association has complicated his posthumous reputation. He remains little known outside Germany, a painter whose Black Forest landscapes speak to regional identity with an honesty that the political appropriation could not quite destroy.