I think brown is an indication of stress. I'm not familiar with Anoles, but I do know the ones I caught when I was in Florida sure turned an icky poo-brown to show they weren't too thrilled with me, lol.
What's your viv set up like? Heat, humidity, how big is the enclosure? I would assume if it's too small, he could be stressed by the presence of the female too close to him, maybe there's not enough foliage and hiding places within it to make him feel secure.
Again, I'm not sure with anoles, I just know they're pretty stress-prone lizards if not kept well, there's likely some people on here who'll know better than I would, I'm just running through the basic things I figure could be causes.
