Hmm, I understood that the issue was the Buffer data structure, and not using an array in and of itself, but yes, it is an array.
So in the same line of thinking, as for me to remove the array altogether was a super change in my mesh building process, instead, assuming it was a java issue, I proceed to call System.gc(), just to test it was unused memory wainting to be cleaned, and it turned out it was.
But again, thanks for the tips and help improve the loading process without consuming much memory in one step, and on how to use the glMapBuffer.
Regards,