Removing porcelin/ceramic towel bar?

You know those bulky towel racks w/the plastic bar and t.p. holders that builders usually install, how are they attached and how do you remove them w/the least amount of damage to the drywall?I know I wil have to patch the wall, but don't want to rip a huge hole in it...