Edwards said he didn't want to "shy away" from speaking to his constituents, who then showed up in force to his town hall to protest and boo him.