Brent Sherwin's 77th-minute drop goal brought Hull's perfect start to the new season to an abrupt end as Castleford turned the form book upside down at the KC Stadium.
Richard Agar's side appeared to be on course for a sixth win in six matches after four tries shared between the Horne brothers Richard and Graeme had given them an 18-12 lead.
But with the usually reliable Danny Tickle failing with three of his four conversion attempts, Castleford were never out of the game and Ryan McGoldrick's try, coupled with Sherwin's drop goal, earned them the win.
It was credit to Terry Matterson's men that they secured a fourth win of the campaign but Agar will surely look upon this as two points dropped in his side's bid to banish the memories of their awful season in 2008, when they finished second bottom, one place above Castleford.
Having won at Bradford for just the second time in the Super League era last weekend, Hull started brightly with a try in the opening minute from Richard Horne but were soon pegged back and only led at the interval after a score from Graeme Horne.
All three tries in an enterprising first half came from errors fielding high kicks and Richard Horne was the first to capitalise.
Ex-Hull winger Kirk Dixon made a mess of taking in Chris Thorman's bomb and in the ensuing melee, Richard Horne reacted the quickest to poach his fifth try of the year on his 250th appearance for the club.
Tickle converted but Dixon was soon able to make amends with a six-point haul after taking in a James Evans pass to ground in the corner after Craig Hall had flapped at Rangi Chase's towering punt. Dixon converted from the touchline.
Hull's ball retention belied their league position and they were lucky to be level after Michael Shenton failed to hold the ball with the line at his mercy after McGoldrick crafted an opening for him.
Hull capitalised on their good fortune and went back in front on 20 minutes.
Richard Owen was the culprit under a high ball this time, failing to clean up another Thorman kick, with Graeme Horne on hand to kick ahead to score.
Tickle's failed goal and Hall's failure to make the most of an empty field ahead of him meant Hull went in at the break with a fragile 10-6 lead.
That advantage was shattered soon after the interval as a combination of Joe Westerman's try and Dixon's goal snuck the visitors ahead for the first time.
Westerman's third try of the season, five minutes after the restart, came off a fine flat pass from Chase, but Castleford's lead only lasted four minutes as Hull struck back.
Graeme Horne was the scorer again, although he did need two attempts to touch down under the attentions of Shenton.
Tickle hit the top of the post with his conversion but it did not seem to matter when the younger Horne secured his hat-trick two days shy of his 24th birthday after good work from Shaun Berrigan.
But with Tickle again failing to convert the scores were left at 18-12 and that was quickly addressed as McGoldrick ran in on an angle to score for the visitors, with a third Dixon goal levelling the scores.
Richard Horne then failed with a 67th-minute drop goal, as did Sherwin five minutes later, before the wily Australian found his aim three minutes from time.