xna - 2D graphics - why use spritesheets?

I have seen many examples of how to render sprites from a spritesheet but I havent grasped why it is the most common way of dealing with sprites in 2d games.

I have started out with 2d sprite rendering in the few demo applications I've made by dealing with each animation frame for any given sprite type as its own texture - and this collection of textures is stored in a dictionary. This seems to work for me, and suits my workflow pretty well, as I tend to make my animations as gif/mng files and then extract the frames to individual pngs.

Is there a noticeable performance advantage to rendering from a single sheet rather than from individual textures? With modern hardware that is capable of drawing millions of polygons to the screen a hundred times a second, does it even matter for my 2d games which just deal with a few dozen 50x100px rectangles?

The implementation details of loading a texture into graphics memory and displaying it in XNA seems pretty abstracted. All I know is that textures are bound to the graphics device when they are loaded, then during the game loop, the textures get rendered in batches. So it's not clear to me whether my choice affects performance.

I suspect that there are some very good reasons most 2d game developers seem to be using them, I just don't understand why.

Answer

A strong argument for using spritesheets is that the number of available textures on a graphic card can be limited. Therefore your graphics library would constantly have to remove texture and re-allocate textures on the GPU. It's much more efficient to just allocate a large texture once.

Also consider that texture sizes are usually power of 2. So if you have a 50x100px Sprite, you'll allocate textures with the size 64x128px or in the worse case 128x128px. That's just wasting graphics memory. Better pack all the sprites into a 1024x1024px texture, which would allow 20x10 sprites and you'll only lose 24 pixels horizontally and vertically. Sometimes even sprites of different sizes are combined into one huge sprite-sheet to use the texture as efficient as possible.

Addendum: A very important reason to use sprite-sheets is to reduce the amount of draw-calls on your GPU, which can have a notable impact on performance. This has been stated in other answers and I'm adding this for the sake of completeness so that this gets some more visibility.