Question

I need to compute a rotation matrix from a direction vector,
and a direction vector from a rotation matrix.

The up direction should correspond to the z-axis,
forward is y and right is x;

    D3DXMATRIX m; // the rotation matrix
    D3DXVECTOR3 v; // this is the direction vector wich is given
    D3DXVECTOR3 r; // resulting direction vector

    float len = D3DXVec3Length(&v); // length of the initial direction vector

    // compute matrix
    D3DXMatrixLookAtLH(&m, &v, &D3DXVECTOR3(0,0,0), &D3DXVECTOR3(0,0,1));

    // use the matrix on a vector { 0, len, 0 }
    D3DXVec3TransformCoord(&r, &D3DXVECTOR3(0,len,0), &m);

Now, the vector r should be equal to v, but it isnt.
What exactly do I have to do to get the results I need?

Answer 1

The question is a bit confusing for me. Using a direction vector, you can define infinite rotation matrices that would make the camera point/look at that direction, unless you define some additional constraints.
I will assume, based on your code, that you want the up vector to be Z 🙂

In first place, be careful: it seems that you are describing a right-handed coordinate system, but using a left-handed function D3DXMatrixLookAtLH.
In second place, reference for this function says that:

D3DXMATRIX* D3DXMatrixLookAtLH(
  _Inout_  D3DXMATRIX *pOut,
  _In_     const D3DXVECTOR3 *pEye,
  _In_     const D3DXVECTOR3 *pAt,
  _In_     const D3DXVECTOR3 *pUp
);

You are creating a look-at matrix specifying that:

• The camera is located at the end of the v vector,
• Camera is looking towards [0 0 0]

Are you sure this is right, or were you maybe looking for the opposite?

In third place, you are applying the rotation matrix to a scaled Y basis vector [0 length 0]. If you want the rotated vector to be v, I think you should apply it to the scaled X basis vector instead [len 0 0].