The mustardy yellow building is Culross Palace which dates from 1597. This is the colour it would have been originally!
You might recognise it as the location for a few films, among them - The Little Vampire and The 39 Steps.
The village is looked after by the National Trust for Scotland who have worked hard to retain the village's historic atmosphere. If you are ever travelling in Fife, you must visit Culross.