I think it's a combination of "it's a tourist thing, and we ate there when I was young..."
I agree, neither of the above is my must go to's. Then again I've been there, done that, bought the Xmas ornament,
Much like places here that get a lot of social media recommendations. They are OK, not great. I include the Colonial Williamsburg Taverns as well as the hotel, The taverns (Kings Arms and Christiana Campbells) are for the novelty, the food is good but not great. The couple of times we've dined at the Hotel, I found the food to be mediocre, but it's also buffet style. There are several places nearby that I prefer.