Drew Barrymore has gone from child star to rebellious ... was that I have had sex with women who work for me on this show,” Letterman said on air. “Now, my response to that is, yes, I have.