2d - rotation direction

I can't understand, which direction (clockwise or anticlockwise) I should rotate my object, after I apply force on it.

So, I have rectangle ABCD (A(1,1);B(-1,1);C(-1,-1);D(1,-1)), and I apply force V(1,1) to vertex D. Of course it should rotate anticlockwise, but what kind of algorithm should I use to define it? Think I already define object speed and direction, rotation speed, but I can't handle with direction.

Answer

You can determine the direction (and magnitude) of a torque by computing the cross product between the vector from the pivot to the point of application and the vector representing the force.

So, in your example:

pivot = (0, 0)

point of application = (1, -1) - pivot = (1, -1)
force = (1, 1)

Since all our vectors are in the 2D plane z = 0, we know our torque points along the z axis so we can compute just its z component:

torque = point.x * force.y - point.y * force.x
       = 1 * 1 - (-1) * 1
       = 2

So we have a positive torque about the z axis, corresponding to a counter-clockwise rotation in the xy plane. If the torque comes out negative, then the resulting rotation will be clockwise instead.