Being a SEAL isn't really that rare. They're too romanticized in public belief. Although they are incredibly badass.
I am going to be a United States Navy SEAL (unless some physical condition precludes me from doing so). I am going to join the Marines out of high school and serve 4 years there, then I should be able to transfer and attempt BUD/S training.
Also just a side note, every time i tell some one of my dream job they always seem to get this mood of "yeah sure". I sometimes wonder if it's just me or do people think that anyone they know couldn't possibly be that tough/elite.
If I were to pursue a military career (I'd thought I did at one time, but other things have convinced me that that's a terrible idea), I'd shoot for Marine Force Recon. 9-man squads that have been known to hold back armies.
While I don't know if you are right about being a SEAL being all that rare I do know that there is an 80% drop out rate. Also about the 9-man squad thing if you haven't read Lone Survivor by Marcus Lutrell i would highly recommend it. Though they of course can't be certain I think the estimated stats are that a 4-man SEAL team (3 of which died) held back close to 200 men and killed somewhere in the ballpark of 40-80. The 200 i got from the book itself, but I forget where I found the 40-80 stats. But in conclusion, while SEAL's are bad [wagon] so are all other branches.