"I apologise, " James said. "You show me nothing but decency and delightful company and yet when you mention... Well. I find myself rather looking past it, but I understand it is who you are. You shouldn't have to hide it."
Other options:
no subject
"I apologise, " James said. "You show me nothing but decency and delightful company and yet when you mention... Well. I find myself rather looking past it, but I understand it is who you are. You shouldn't have to hide it."