Can Geometric Deep Learning lead eliminate the need of "Brute Force" pre-training [D]

I’ve been reading about Geometric Deep Learning lately (the whole grids, graphs, groups, manifolds idea), and something clicked that i wanted to get a clarity on, i don't think i'm an expert at GDL or anything mentioned here, so i can most definitely be wrong at a fundamental level as well,

A lot of modern deep learning feels like we're throwing massive data and compute and we just hope the model learns the right invariances.

But doesn't GDL kind of flips that?

Instead of learning invariances (like rotation, permutation, etc.), you can build them directly into the architecture using symmetry and geometry. So it got me wondering, if a model literally cannot break a symmetry (like confusing a rotated cat for something else), does it even need tons of examples to learn that, Like why show it 10,000 rotated cats if rotation invariance is already guaranteed?

Which leads to a bigger question:

Are we doing massive-scale pretraining mostly because our architectures are missing the right inductive biases, And if we get the geometry right, does the need for huge datasets actually go down?

it feels like a shift from learning everything from the data to encode what must be true, learn the rest to me

still haven't read the recent advancements in GDL to comment enough, thought i should ask experts here