St. Jacob’s Chapel, the oldest building in Osterspai Castle, is believed to have been presented to the monastery of Eberbach in 1236. The knights of Liebenstein, who took possession of the castle at the end of the 13th century, built the residential tower in the mid-14th century. Four of the towers' stories still exist today. Remnants of the former corner towers are still recognisable directly under the roof.

The barons of Waldenburg became masters of the moated castle of Osterspai on the banks of the Rhine in the 17th century. At the end of the 18th century, it became a possession of the barons of Preuschen. In 1910, a half-timbered house was built next to the residential tower for use as a summer residence. Ten years later, it was extended for permanent residential use.

The Chapel of St Jacob is a two-story rectangular hall. On the vaulted ceiling of the upper floor are traces of 13th century murals stylistically reminiscent of those in Limburg Cathedral. Restoration work was completed in 1995. Visits by arrangement only.