The reason modern games have a terrible bang-per-buck when it comes to VRAM usage is because in D3D11 and earlier there were 128 shader resource slots max, and so for every draw call the system knows which textures it will reference. OS would page textures in and out.
Then came D3D12 and bindless APIs, and the developers were in charge of this process, known as residency management. Almost everyone seems to have dropped the ball here, and they just make all textures resident all the time in VRAM, and you end up with this...
The games certainly have enough info about which textures might be referenced by which drawcalls, but without actively informing the OS about this via Evict() / MakeResident() everything stays in VRAM all the time and artists end up lowering texture res to hit perf targets.
It seems that the game programmer answer to "how do you fit a large dataset in memory" is "buy more memory lol". We're all suffering as a result.