It's a sensitive subject, sort of. Cole has a soft spot for stray animals, those that have been abandoned and kicked to the side. The fact that they tolerate him more here than in Skyhold is just a bonus. He must smell different or something, more like a real boy. His presence doesn't scare them, and he cherishes that and holds it closely to his chest.

"You scared them," he says, crossing his arms against his chest. "You scared them and you didn't have to do that. And now they might not ever come back."

He's trying to rein in his anger. Taking deep breaths is helping, just like Varric taught him.