He previously played Dr. Johnathon Leroy in a recurring role in East Enders, and he played Bryan in The Green Green Grass. He also played the Duke of Milan in all three seasons of The Borgias.
His passion entertainment started early. He recreated television adverts at age two and performed shows with his friends throughout his childhood. He joined a youth theatre group at age eleven. Due to his parents' jobs as social workers, Kaye attended thirteen different secondary schools and experienced bullying. He made his first West End theatre appearance in 1980 as Willie Wonka in a stage version of Charlie and the Chocolate Factory.
- Kaye says his Russian ancestors claimed to be descended from the Vikings. Clive Standen (Rollo) described Kaye in an interview in 2015 as the most Viking-like cast member in real life, stating, "He's got Viking in him."