Caballero started his playing career with Boca Juniors in 2001, where he won three major titles in 2003.
Caballero joined Elche CF of Spain in 2004, and had a brief loan spell back in his country with Arsenal de Sarandí in 2006. With the former, after a rough first season, he went on to become the undisputed starter, appearing in nearly 200 Segunda División games.
On 10 February 2011, Caballero was transferred to Málaga CF in La Liga for €900,000 and two and a half years, as an emergency transfer – after the 31 January transfer deadline – due to a serious knee injury to Sergio Asenjo (the club's previous starter, Rubén, was also unavailable due to physical problems); he made his league debut on 20 February, starting in a 1–1 draw at Villarreal CF, and played all the matches until the end of the campaign, with the Andalusians finally escaping relegation.
On 8 July 2014, Caballero signed a three-year deal with Manchester City, for a fee believed to be in the region of £6 million excluding add-ons. He reunited at his new club with former Málaga manager Manuel Pellegrini,and his competitive debut came on 10 August in the 2014 FA Community Shield, playing the full 90 minutes in a 0–3 loss to Arsenal at Wembley Stadium; a week later, in the Premier League opener, he was an unused substitute at Newcastle United
Caballero played for Argentina at under-20 level, helping the nation win the 2001 FIFA World Youth Championship by playing in the competition's last two games.