CableGuy,
You had a similar problem to this with your pairs game and as I recall I made you I snippet of code (sorry but I've lost it now

) whereby I placed all the image names into a String separated with commas.
All that had to be done was randomly select any point in the String and find the first comma then remove the image name and add it into the desired position in your grid. The string is made shorter because the entry was removed and so as LineCutter put it, there's NO futile loops!
Sounds like you need a similar thing with your Array of numbers. If you want the code snippet I think it probably best to cut and paste from your pairs game.
Regards,
RandomCoder.