300+ population, no raids yet, is this normal?

From various YouTube videos and comments on this subreddit, I was expecting raids to come when my pop hit 100. Maybe it's outdated info (I'm running v66 here).

When I hover over the army/number of soldiers button it does say I am not at risk of raids yet. I currently have around 80 melee soldiers, 40k denari, is this perhaps enough to "scare" away the raids for now? Not sure what's going on.

Also, I have no walls or gatehouses etc... When raids do finally come, are they going to hurt?