Currently, there is so much news about Petraeus' affair that it's easy to overlook the fact that there were divisions between him and rank and file CIA personnel before the affair was ever revealed.
For example, Benghazi caused a significant amount of strife, because of the way Petraeus handled it, as did his domineering style of okaying and prohibiting drone strikes and other intelligence exercises. His style appears to have been a reflection of the fact that he was a square peg in a round hole at the CIA, where respect and trust do not come easy.
Regarding Benghazi, after the public relations disaster began to emerge in the days after September 11, Petraeus urged CIA personnel to "push back hard" and release their own timeline of the attack on the consulate and the "CIA safehouse." Petraeus' goal was to paint the CIA in a more positive light.
National Security Director James Clapper disagreed with this approach, as did the Pentagon. Yet Petraeus forged on and urged his aides to forge on as well.
Regarding the strained relations at the CIA which preceded Benghazi or news of Petraeus' affair, it seems few people at the CIA were impressed with Petraeus' military rank. In fact, they were somewhat turned off by the fact that he handled his CIA position in a way befitting the military instead of an intelligence agency.
Various reports indicate Petraeus would be bothered if his orders were met with questions or if "young analysts would disagree with a point he made." In short, there was very little give and take.
Some say it seemed like Petraeus stepped from the battlefield to intelligence headquarters without checking his ego at the door.