If you have ever found a tomato growing in your rose bed, you know that many vegetables can be grown from produce at your grocery store, at a much lower cost. Here is a list of a few: http://gardeningjones.com/blog/?p=83
As for potatoes, I have grown Certified Seed Potatoes ("Certified Seed" potatoes are more resistant to disease) and I have grown potatoes left from the previous year's crop. I have also grown potatoes I bought at the grocery store.
I cannot say one was better tasting or producing than another. The cheapest of course is what is left from the previous year's crop. For us, there is rarely any left!
I can say the potatoes from the grocery were significantly less expensive than those from the seed catalogs. Try it yourself and decide.
How to: look for potatoes that show a little sprouting. All you need to do is cut the potato so that each piece has 3-5 eyes. Let sit a bit so the edges begin to dry, this helps prevent rotting when you plant them. Note that each piece might produce from 3 to 6 potatoes.
Plant after danger of frost in well-composted soil. Be sure to keep piling rough compost and/or straw as the plants grow, to make sure the potatoes do not see the sun (it will turn them green).
You can 'grapple' early or 'new' potatoes soon after you see flowers. This is a method of carefully picking a few small potatoes without disturbing the entire crop.
Decide for yourself which type of potato to plant based on your trials and considering flavor and storage.