I've never actually found a good resource for studying Icelandic. They're all crappy for one reason or another.
I would suggest, however, the Hippocrene dictionary and Teach Yourself Icelandic (the one by PJ Glendenning). The latter has been highly criticized, but I've actually found that if you go through it and make some organized notes (I was able to condense all my notes into a useful 2 pages front-and-back, in fact), it's a great grammar resource. Other notes from the book are online at
www.samkoma.com/mimir/mimir.htm if you're hurting for money, but I would suggest you obtain a copy for yourself -- Mímir leaves out a lot of explanation and irregularities.
http://icelandic.hi.is is also decent, but I've never personally gotten too far in it. Probably because it's more practical if you're planning on going to Iceland than if you're planning to read Icelandic.
But here's the thing -- Icelandic is pretty easy to practice. For some reason, Icelanders are all over the internet, and Iceland is famous for its incredible output of culture (Björk, Sigur Rós, Lazytown, etc., among the fact that Iceland publishes more per capita than any other country). They are very worldly too. I first met Icelanders while in the West Bank last summer.