Gillingham manager Justin Edinburgh questioned whether Oldham's two goals should have stood as they roared back to claim a first win since August.
The home side looked to be heading for maximum points after Cody McDonald's opener.
But substitute Freddie Ladapo equalised on 65 minutes before Ryan Flynn turned the result around with 60 seconds of normal time remaining.
Edinburgh complained: "We started brightly and looked in charge of the game but in the last 15 minutes of the first half we retreated and that allowed Oldham to develop their game.
"I believe that the two goals should be disallowed. I think the first goal is a foul on Chris Herd and the second one there was a player obstructing (goalkeeper) Stuart Nelson from an offside position.
"Oldham were a better team in the second half but we had some key decisions go against us.
"Our mindset was to protect rather than take the game out of Oldham's hands.
"I thought Frank (Nouble) played really well but he ran out of steam. Chris Herd also played really well but he got quite a nasty dead leg and had to come off.
"We've got quite a few players coming back from injuries so hopefully in the next few weeks I'll be able to work out what my strongest team is."
The Latics went into the game at the foot of the League One table, but got back on track after ending a run of seven games without success.
And manager Steve Robinson was delighted with the character shown by his team not to throw in the towel when the going got tough.
Robinson said: "At one down you're fighting against the odds but we showed a lot of character.
"To come here and get three points was an incredible result.
"We had chance after chance and sometimes this can let the opposition back into the game but credit to the team, they stuck at it.
"Freddie took his goal really well. He came on and put us into life. He was tired after midweek when he put in a good shift so we knew he couldn't do a full 90 but he was great off the bench.
"After 23 minutes we had our first pattern of play, string a few passes together, and after that the belief started coming.
"Next week (a local derby against Bolton) is going to be a big game but we will take confidence from today and if we perform like we did in the second half then I don't see why we can't get another win.
"I knew this was going to be a hard job. It does take time but I'm starting to settle."