glsl - How can I reliably implement GPU skinning in Android?

I'm trying to get character skinning working on Android.

The idea is quite vanilla: I have my skinning matrices, and along with each vertex, I send up to four matrix indices and four corresponding weights. I sum them in the vertex shader, and apply them to each vertex.

This is what I'm doing in the vertex shader in the iOS version of my game (don't mind the normals):

attribute vec4 in_pos;
attribute vec4 in_normal;
attribute vec2 in_texture_coords;
attribute vec4 in_bone_index;
attribute vec4 in_bone_weight;


varying vec2 fs_texture_coords;

uniform mat4 world_view_projection;
uniform mat4 bones[@bind_matrix_count];

void main()
{
    // Skinning
    vec4 transformed_pos =
        ((in_pos * bones[int(in_bone_index.x)]) * in_bone_weight.x) +

        ((in_pos * bones[int(in_bone_index.y)]) * in_bone_weight.y) +
        ((in_pos * bones[int(in_bone_index.z)]) * in_bone_weight.z) +
        ((in_pos * bones[int(in_bone_index.w)]) * in_bone_weight.w);

    gl_Position = world_view_projection * transformed_pos;
    fs_texture_coords = in_texture_coords;
}

And it works pretty well. However, with the same code in Android, in some devices (notably the Nexus 7 2013), you can't access uniforms with non-constant indices. In other terms, you can't do this:

bones[int(in_bone_index.w)]

because bones[some_non_constant] is always evaluated as bones[0], which is not amusing at all. The worst thing is that the shader compiler happily compiles this.

This guy seemed to have exactly the same problem. He solved it by accessing the uniforms as vectors instead of matrices. I did the same, and in fact it worked!

attribute vec4 in_pos;
attribute vec4 in_normal;
attribute vec2 in_texture_coords;
attribute vec4 in_bone_index;
attribute vec4 in_bone_weight;

varying vec2 fs_texture_coords;


uniform mat4 world_view_projection;
uniform vec4 bones[@bind_matrix_count * 4]; // four vec4's for each matrix

void main()
{
    // Skinning
    mat4 skin_0 = mat4(
        bones[4 * int(in_bone_index.x) + 0],
        bones[4 * int(in_bone_index.x) + 1],

        bones[4 * int(in_bone_index.x) + 2],
        bones[4 * int(in_bone_index.x) + 3]);
    mat4 skin_1 = mat4(
        bones[4 * int(in_bone_index.y) + 0],
        bones[4 * int(in_bone_index.y) + 1],
        bones[4 * int(in_bone_index.y) + 2],
        bones[4 * int(in_bone_index.y) + 3]);
    mat4 skin_2 = mat4(
        bones[4 * int(in_bone_index.z) + 0],
        bones[4 * int(in_bone_index.z) + 1],

        bones[4 * int(in_bone_index.z) + 2],
        bones[4 * int(in_bone_index.z) + 3]);
    mat4 skin_3 = mat4(
        bones[4 * int(in_bone_index.w) + 0],
        bones[4 * int(in_bone_index.w) + 1],
        bones[4 * int(in_bone_index.w) + 2],
        bones[4 * int(in_bone_index.w) + 3]);
    vec4 transformed_pos =
        ((in_pos * skin_0) * in_bone_weight.x) +
        ((in_pos * skin_1) * in_bone_weight.y) +

        ((in_pos * skin_2) * in_bone_weight.z) +
        ((in_pos * skin_3) * in_bone_weight.w);

    gl_Position = world_view_projection * transformed_pos;
    fs_texture_coords = in_texture_coords;
}

But I think this worked as chance. uniforms are not meant to be randomly accessed, so I fear this "technique" will not work in every device.

This guy is passing his matrices as textures, which is a pretty cool idea. I made a 4x32 OES_texture_float texture, where each texel is a matrix row, and each texture row is an entire matrix. I access it like this:

attribute vec4 in_pos;

attribute vec4 in_normal;
attribute vec2 in_texture_coords;
attribute vec4 in_bone_index;
attribute vec4 in_bone_weight;

varying vec2 fs_texture_coords;

uniform mat4 world_view_projection; // A texture!
uniform sampler2D bones;


void main()
{
    // Skinning
    mat4 bone0 = mat4(
        texture2D(bones, vec2(0.00, in_bone_index.x / 32.0)),
        texture2D(bones, vec2(0.25, in_bone_index.x / 32.0)),
        texture2D(bones, vec2(0.50, in_bone_index.x / 32.0)),
        texture2D(bones, vec2(0.75, in_bone_index.x / 32.0)));
    mat4 bone1 = mat4(
        texture2D(bones, vec2(0.00, in_bone_index.y / 32.0)),

        texture2D(bones, vec2(0.25, in_bone_index.y / 32.0)),
        texture2D(bones, vec2(0.50, in_bone_index.y / 32.0)),
        texture2D(bones, vec2(0.75, in_bone_index.y / 32.0)));
    mat4 bone2 = mat4(
        texture2D(bones, vec2(0.00, in_bone_index.z / 32.0)),
        texture2D(bones, vec2(0.25, in_bone_index.z / 32.0)),
        texture2D(bones, vec2(0.50, in_bone_index.z / 32.0)),
        texture2D(bones, vec2(0.75, in_bone_index.z / 32.0)));
    mat4 bone3 = mat4(
        texture2D(bones, vec2(0.00, in_bone_index.w / 32.0)),

        texture2D(bones, vec2(0.25, in_bone_index.w / 32.0)),
        texture2D(bones, vec2(0.50, in_bone_index.w / 32.0)),
        texture2D(bones, vec2(0.75, in_bone_index.w / 32.0)));
    vec4 transformed_pos =
        ((in_pos * bone0) * in_bone_weight.x) +
        ((in_pos * bone1) * in_bone_weight.y) +
        ((in_pos * bone2) * in_bone_weight.z) +
        ((in_pos * bone3) * in_bone_weight.w);

    gl_Position = world_view_projection * transformed_pos;

    fs_texture_coords = in_texture_coords;
}

In fact, this worked pretty nice... Until I tried it on my Galaxy Note 2. This time the compiler complained that I can't use texture2D on the vertex shader!

So what I'm doing is checking whether the GPU supports texture accesses on the vertex shader and if it supports OES_texture_float. If it does, I'm using the texture approach. If it doesn't, I'm using the vector approach.

However, the texture approach is not available on all platforms, and the vector approach is kinda working by chance. I would like to know if there is a way to pass my skinning matrices to the vertex shader, which reliably works on all devices.

I can have minimum reasonable OS requirements, like Android 4.1+, but I would like to have a solution that works on all devices which meet those requirements.

Answer

This is non-conforming behaviour by the Nexus 7 (Adreno GPU). You say "uniforms are not meant to be randomly accessed", but according to Appendix A of the spec:

Uniforms (excluding samplers)

In the vertex shader, support for all forms of array indexing is mandated. In the fragment shader, support for indexing is only mandated for constant-index-expressions.

It sounds from the discussion here that this bug applies only to uniform matrix arrays, so the work-around using vectors is likely to work reliably and be portable to other GPUs (I know random uniform indexing works at least on Mali and PowerVR GPUs).