Best way to interpolate player movements in a very fast paced Unity game?

In my Unity online multiplayer game players move very fast and I currently use Lerp to interpolate between their last position and current position:

transform.position = Vector3.Lerp(transform.position, this.correctPlayerPos, Time.deltaTime * smoothingFactor);

smoothingFactor is currently around 35f but I don't like the result. If I lower smoothingFactor, movements will be smoother but players will lag as well. If I increase it to like 40, it will be more precise but laggy.

How can I solve this?