I was gonna say something like this too, because we had an indoor-outdoor cat when I was a teenager and I'm sure at least some of the neighbors thought he was horribly neglected. He had all his vet care done and had every opportunity to be a spoiled indoor cat but that is NOT what he wanted. He would find a way out. He roamed far and wide and hunted and fought and even joined a nearby church (seriously! He was in their church directory and everything) and was there every Sunday. I suspect some of the neighbors even tried to keep him indoors and soon found out that wasn't going to happen. He was his own cat and lived on his own terms.My poor outside cats are out in a hundred-degree heat every day. I even water down the lilacs near our place to give them some cool dirt to lay on, but they prefer being in a metal garage! Mama has 4-week old kittens in there, too. I have two big pans of water in the shade of the building and when it got so hot I even have a box fan in the garage to stir things around. Cats are desert animals so I pray they aren't as hot as I am!
And if they don't have AC it's probably no better indoors; at least outside he can find a cool bush to sleep under.
So, I honestly don't think Butters is being neglected, and even if they agree to let him move with you, you'll probably find out quickly how much he doesn't want to be inside all the time. Yeah, it stinks that it gets your cats all in a tizzy and is against the city codes, but some cats are just like that.