opengl - Why do we use 4x4 matrices to transform things in 3D?

To translate a vector by 10 unit in the X direction, why do we have to use a matrix?

We can just add 10 to the mat[0][0], and we got the same result too.

Answer

Yes, you can add a vector in the case of translation. The reason to use a matrix boils down to having a uniform way to handle different combined transformations.

For example, rotation is usually done using a matrix (check @MickLH comment for other ways to deal with rotations), so in order to deal with multiple transformations (rotation/translation/scaling/projection...etc) in a uniform way, you need to encode them in a matrix.

Well, more technically speaking; a transformation is mapping a point/vector to another point/vector.

p` = T(p); 

where p` is the transformed point and T(p) is the transformation function.

Given that we don't use a matrix we need to do this to combine multiple transformations:

p1= T(p);

p_final = M(p1);

Not only can a matrix combine multiple types of transformations into a single matrix (e.g. affine, linear, projective).

Using a matrix gives us the opportunity to combine chains of transformations and then batch multiply them. This saves us a ton of cycles usually by the GPU (thanks to @ChristianRau for pointing it out).

T_final = T * R * P; // translaterotateproject

p_final = T_final*p;

It's also good to point out that GPUs and even some CPUs are optimized for vector operations; CPUs using SIMD and GPUs being data driven parallel processors by design, so using matrices fits perfectly with hardware acceleration (actually, GPUs were designed to fit matrix/vector operations).