Why are textures always square powers of two? What if they aren't?

Why are the resolution of textures in games always a power of two (128x128, 256x256, 512x512, 1024x1024, etc.)? Wouldn't it be smart to save on the game's file size and make the texture exactly fit the UV unwrapped model?

What would happen if there was a texture that was not a power of two?

Would it be incorrect to have a texture be something like 256x512, or 512x1024? Or would this cause the problems that non-power-of-two textures may cause?

Answer

Why are the resolution of textures in games always a power of two (128x128, 256x256, 512x512, 1024x1024, etc.)?

As Byte56 implied, the "power of two" size restrictions are (were) that each dimension must be, independently, a power of two, not that textures must be square and have dimensions which are a power of two.

However, on modern cards and which modern graphics APIs, this "restriction" has been relaxed significantly such that textures dimensions can be, within reason, just about anything you like. However:

Wouldn't it be smart to save on the game's file size and make the texture exactly fit the UV unwrapped model? What would happen if there was a texture that was not a power of two?

By ensuring the texture dimensions are a power of two, the graphics pipeline can take advantage of optimizations related to efficiencies in working with powers of two. For example, it can be (and absolutely was several years back before we had dedicated GPUs and extremely clever optimizing compilers) faster to divide and multiply by powers of two. Working in powers of two also simplified operations within the pipeline, such as computation and usage of mipmaps (a number that is a power of two will always divide evenly in half, which means you don't have to deal with scenarios where you must round your mipmap dimensions up or down).

It's true you "waste" some space this way, but the extra space is usually worth it for the tradeoff in render performance. Additionally there are techniques, such as compression or packing multiple images into a single texture space that can alleviate some of the storage waste.