opengl - Voxel Game - Lag when camera is inside of the terrain

I am making a voxel game in OpenGL and I currently render 8 x 8 chunks to the screen. The first thing I do is generate the map using Simplex Noise and this is done once when the chunk is created. I store each block in a 3D array contained in each chunk. After the chunk is created I then call a function that checks each block to see if it is next to an air block, if it is then it is added to a separate array of visibleBlocks as is, otherwise an empty air block is added to its place (There is probably a better way of doing this). Once I have an array of blocks of which are visible, I call a function that returns a mesh which holds all of the vertices, colour data, normals, etc into 3 big VBOs (one for each) rather than rendering each block to the screen with its own vbo (of which I couldn't even render one chunk properly).

public static Mesh genChunkMesh(Block[][][] visibleBlocks)
{

    ArrayList vertexData = new ArrayList(); //Storage for the chunk vertices
    ArrayList colourData = new ArrayList(); //Storage for the colours
    ArrayList indexData = new ArrayList(); //Storage for the indices
    ArrayList normalData = new ArrayList(); //Storage for the normals

    int blockNum = 0; 
    for(int x = 0; x < CHUNK_WIDTH; x++)
    {
        for(int y = 0; y < CHUNK_HEIGHT; y++)
        {

            for(int z = 0; z < CHUNK_WIDTH; z++)
            {
                //Grab the mesh for a given BlockType from some statically defined blocks
                Mesh blockMesh = Chunk.getBlockMesh(visibleBlocks[x][y][z]);

                if(blockMesh != null)
                {
                    float[] vertices = blockMesh.getVertices();
                    float[] colours = blockMesh.getColours();
                    int[] indices = blockMesh.getIndices();

                    float[] normals = blockMesh.getNormals();

                    for(int i = 0; i < vertices.length; i += 3)//Add the next x, y, z position to the vertex data
                    {
                        vertexData.add(vertices[i] + visibleBlocks[x][y][z].getPosition().x);
                        vertexData.add(vertices[i + 1] + visibleBlocks[x][y][z].getPosition().y);
                        vertexData.add(vertices[i + 2] + visibleBlocks[x][y][z].getPosition().z);           
                    }

                    for(int i = 0; i < colours.length; i++)

                    {
                        colourData.add(colours[i]);
                    }

                    for(int i = 0; i < indices.length; i++)
                    {
                        indexData.add(indices[i] + (blockNum * 24)); //Indices range from 0-23, so every block 24 is added to each index
                    }

                    for(int i = 0; i < normals.length; i++)

                    {
                        normalData.add(normals[i]);
                    }
                    blockNum++;
                }
            }
        }
    }

    return new Mesh(Util.toFloatArray(vertexData), Util.toIntArray(indexData), Util.toFloatArray(colourData), Util.toFloatArray(normalData), grassMaterial);

}

I have a world object which holds these 8 x 8 chunks (the reason for 8 x 8 is because if I do anymore it begins to lag and each chunk being 16 * 16 * 128 blocks. Also I am implementing infinite terrain but at the moment I want to make sure a finite terrain of 8 x 8 works first). Each chunk has its own mesh which itself has one vao and 3 vbos (vertex, colour, normal) which is only generated once when the chunks are created and eventually will be regenerated everytime the chunk is updated (haven't got to this yet). In my render method the chunk array is looped through and each mesh is rendered through:

    //Bind the VAO
    glBindVertexArray(vaoID);
    glEnableVertexAttribArray(0); //Vertex Data
    glEnableVertexAttribArray(1); //Colour data
    glEnableVertexAttribArray(2); //Normal data

    glDrawElements(GL_TRIANGLES, numVertices, GL_UNSIGNED_INT, 0);


    //Restore state
    glDisableVertexAttribArray(0);
    glDisableVertexAttribArray(1);
    glDisableVertexAttribArray(2);
    glBindVertexArray(0);

Now that you have some context, my question is, why do I get consistent 60 fps when I am outside of the world

However when I move inside a chunk below the terrain, I get stuttering frames that jump around 30 fps.

I am unsure of what I am doing wrong that is causing this to happen, I have been searching for similar problems online but I do not know how to word it as it is weird. Also as you can see in the first image, weird lines appear at far distances between the blocks. I have a directional light that is pointing straight down and I don't know if it is because I am rendering faces between two solid cubes and the top face is brighter than the side faces. Another thing I should mention is that glfwSwapBuffers(1) is called at the end of each render call and it is causing a considerable amount of time to finish (around 0.01 to 0.05 seconds per call).

tldr: As shown in the pictures, I get consistent 60 fps when outside/above the terrain, but when I am inside of the terrain, the fps stutters around 20-30 fps and I don't know why.