I grew up about a mile from MBS and now I'm near Selfridge air base. As you say, it's quite noticeable when a plane isn't where its 'supposed' to be.
My first flight wasn't until I was 18. My dad flew regularly for work though and I was flying with him. We went through a thunderstorm. I had a window seat and all I could think off was I was going to see that monster from the Twilight Zone on the wing with the next flash of lightning. I was amazed at how much the wings could flex from the turbulence. My dad was calm the entire time and just said most flights aren't this bad, but it's normal.
The next day while visiting my sister, she asked how the flight was, my dad quickly announced it was awful, he thought we might die, he never experienced anything like that in his life, and he was a little afraid to get on a plane to go back home.
The plane ride home was uneventful and I never developed a fear of flying. It would have been wildly different if he expressed how he was really feeling during that flight though!