I had a few moments today and started thinking about your approach, I'm not sure if I understand what you are trying to do completely, so I thought I'd try to implement something like this:
First, I use a 2 dimensional array to indicate if a location is occupied by an object. Then, as my camera moves around, I want the camera to "collide" with the objects using the camera's location as an index into the 2 dimensional array.
This approach would work fine and is very fast also but the camera would be right at the edge of the object. What you are missing is that the camera also has a size.
The simplest approach is to perform some kind of collision detection between the object and the camera, but you want to avoid that. Well, in this case, you must use your camera's location as a center point and compare all 8 objects around your camera in the 2 dimensional case, I have this in my code:
if (map[x-1][z+1] != null || map[x][z+1] != null || map[x+1][z+1] != null ||
map[x-1][z] != null || map[x][z] != null || map[x+1][z] != null ||
map[x-1][z-1] != null || map[x][z-1] != null || map[x+1][z-1] != null) {
// Collision detection here
This way, you detect your collision much sooner. I had a little fun with this and created the following:
http://whalechat.com/ballcraft_demo/You can freely add and remove spheres on the plane using the space bar and the 'R' key.
Hope this helps.