You COG is around the belly button, mid core of your body. As drivers are in a "v" shape in the car, I'd assume their COG is around the physical belly button.
Imagine taking 20kg's from there and putting it below the chair, that's 25 cm lower easy.
Engineers will chew off a limb to get "a small sack of cement" 25 cm's lower in the car.
20kg's is a bit over 2,5% of the total weight of the car. The above lowers the COG by almost 6,3mm
Ballast is not about what you put in the nose of the car like the pretty pictures. Actually, engineers would move any ballast to the center of rotation, as low as possible if they could choose. They want to limit it's inertia ans lower it's COG.
Those picture of ballast in the nose is to compensate for something else (handling or axle weight limits) they are not able to compensate otherwise and will be high on the priority list to move it back to the center of the car.