Amsterdam has a wonderful (so I am told by horsey people) riding school in the Hollandse Manege. The building is based on the famous Vienna riding school.
There are 153 stalls and boxes, originally supplied by the St Pancras Iron Works co, with dark wooden dividing walls and blue and white tiles on the back walls. As at any riding school, a handful of teenaged girls are always there tacking-up. On the first floor is a restaurant.
Threatened with demolition in the 1980s the building was saved by public outcry. In 1986 it was reopened by Prince Bernhard and restored to its former glory. The horse trail in the Vondel park is still in use. I don't know if you can just turn up and borrow a horse, but you can always try.
Vondelstraat 140, website (Dutch only)