Cg, like DirectX's HLSL, compiles to low-level shaders/programs. You can do it offline, or at runtime. The problem is, there are lots of targets (nv20, nv30, ati ones, etc.) to worry about. As an example, I read in a Valve presentation that Half-Life 2 uses THOUSANDS of, HLSL generated, low-level shaders. They write HLSL, compile all of the possible "versions" of the shaders (different # of lights, # of bones, enabled effects, etc.) and use the low-level ones at runtime, which of course is a big mess to manage.
With GLSL, the drivers compile at runtime only. And you don't have to worry about what's being generated. The driver will choose the best one for the hardware to run it. For example, on my FX it compiles to NV VP2.0 and NV FP1.0, thus being able to support everything available on my card (and optimize better). On a GeForce4, it'll probably compile to NV VP1.0 and an ATI would use the ARB programs. But you really don't (and shouldn't) care. It's all being done behind the scenes, you only care about shader objects and shader programs. No low-level stuff. It's much easier for the developer and as I said, if you want more flexibility you can always use Cg stuff (hint: check for the EXT_Cg_shader extension availability).