The herb garden is still relatively young, having only been in existence for around 40 years. However, the horticultural use of this part of the palace garden can already be seen in an old town view from 1777. A smaller building is also depicted, as is a large tree and a path running along the length of the garden. The tree is probably the lime tree still standing today, while the building is the pavilion, which has shone in new splendor since its renovation in 1998. In the 18th century, it was known as the Lusthaus and was used as a "small castle café" in the 1920s and 1930s. In the first half of the 19th century, Prince Georg of Saxe-Altenburg had the palace garden rebuilt in the English style by court gardener Döll. In the course of this change, the summer house was probably rebuilt or converted in a neoclassical style. From 1829, the palace garden was also open to the public.

In the 1960s, the herb garden was finally laid out in its current form. The initiators, Parkaktiv des Schlossgartens, created beds and planted them with a wide variety of herbs. The layout of the beds has been preserved to this day and over 50 different herbs, medicinal and aromatic plants grow here.