unity - How to make sure an object NEVER passes through an edge collider (2D)?

In my top down perspective game, I have a ball that bounces off of walls. The ball should behave in a way where it will always bounce off in a 90 degree angle. The ball can only move up, down, left and right and the walls are slanted so that they should always (theoretically) bounce a ball up, down, left or right.

On the ball object, I have a small circle collider 2D and a Rigidbody 2D attached, the walls have a single edge collider 2D attached. Approximately one time out of twelve times, the ball will simply pass through the edge collider, so it's not consistent.

How can I make sure that the ball always registers the collision?

Here's my setup:

The script I use to move the ball:

void Update () {
    switch (direction)
    {
        case 'u':

            transform.Translate(0f, ballSpeed * Time.deltaTime, 0f);
            break;
        case 'd':
            transform.Translate(0f, ballSpeed * -1 * Time.deltaTime, 0f);
            break;
        case 'l':
            transform.Translate(ballSpeed * -1 * Time.deltaTime, 0f, 0f);
            break;
        case 'r':
            transform.Translate(ballSpeed * Time.deltaTime, 0f, 0f);

            break;
    }
}

Additional info:
- every wall is it's own object with it's own edge collider.
- the ball always hits the center of the wall because I center the ball object using the center of the wall it touches. Note: I had issues with the ball passing through walls even before I implemented the centering.