It was never proven that Bagwell or Pudge took PEDs.
Like you, I think the majority of players took PEDs in the era prior to stricter testing and penalties. So, I don't care about PED use so much during that period.
Once they had strict penalties in place and players like Ramirez and Ortiz still got caught taking them, I can see where they might be judged more harshly. Ortiz got in because the media liked him. Ramirez didn't because they disliked him which of course is unfair.