If it is really a quad, that is, all vertices lie on the same plane, then you just need to take any triangle of the quad and compute the normal of it with a single cross product like so:

`// v2 - v1`

Vector3f v12 = v2.sub(v1, new Vector3f());

// v3 - v2

Vector3f v23 = v3.sub(v2, new Vector3f());

// (v2 - v1) x (v3 - v2)

return v12.cross(v23).normalize();

While conceptually correct (all cross products on the spanning vectors of a quad's vertices should be the same), your code suffers from aliasing between the 'nX' variables, 'a', 'b' and 'n' (i.e. those are all *the same* object).