Mostly the remote but I have and still do on occasion call it the clicker or controller. And when my brain doesn't work...the TV Thingy. The "insert descriptive word" Thingy is used for many items around my house and is usually accompanied by a wagging pointing finger to it or something that is associated it.
For example...for the remote...I would say TV thingy, but either wag my finger at the TV or perhaps where the remote usually sits.