Ultimate Cards are not the same as Special Cards, and if a fighter draws an ultimate but is switched out for another fighter, that ultimate converts to a special.
In game, using an ultimate for challenges will not count as using a special, and the same goes for extras (which in-game challenges refer to as special skills).