2001 New England - at the time Tom Brady was not an "elite" QB. The Patriots defense that year was a Top 10 unit.
I don't think that Brady was elite in 2003 or 04, either.
In 2003, NE's defense was #1 in points allowed with 238, and forced seven turnovers in the playoffs (including five in the AFC Title Game against the Colts. That is the game where Brady should have thrown four INT's) en route to their second title in three years.
That year, Brady had 3,620 yards passing and 23 TD's, but he only had an 85.9 rating.
The next year, the Pats were second in scoring defense (260 allowed, 16.3 ppg) and forced 11 postseason turnovers en route to their third title in four years. Brady was a little bit better that year (3,692 yards passing, 28 TD's, 92.6 rating), but he was aided by a new RB in Corey Dillon (1,635 yards rushing in 04).
In fact, I don't think that Brady is as elite as people make him out to be. He didn't have a rating over 100 until 2007, when Welker and Moss came on the scene (Brady threw for 50 TD's, and Moss caught 23 of them). The year before, with a way worse WR core, he only had an 87.9 rating. That shows me that the voters got the MVP wrong in 2007.
After that, his next year with a rating over 100 was 2010, when they added the two star TE's in Gronk and Hernandez (he also had a rating over 100 in 2011 with those two). By 2013, though, when he had no Welker, no Hernandez, and only seven games of Gronk, he had an 87.3 rating.
In addition, he always seems to have a defense on his side that is in the top-10 in the NFL in points allowed (2002, 05, and 11 are the exceptions).