I think that psychological barriers that make it harder for people to wander into land are the way to go, including brash, encouraging brambles etc, leaving felled tree crowns as obstacles, and light fencing - even 50m of plastic deer fencing stretched between trees across a likely entrance. Ideally, make it so there is only one easy way in and out, as that makes it harder to walk through the wood and raises the apparent risk they'll be forced into a confrontation with you if it turns out you're there. Thieves and yobs aside, most people can be shamed into stopping doing something if they are confronted enough. Even if they are all mouth at the time, most will think again if they anticipate another embarrassing experience next time.
"This is my land. You are disturbing me and the wildlife. You have no business here. This is not on a route to anywhere else. Now go back the way you came in."
We can't build fortresses to keep them out, but as DuncanB has shown, it can be minimised. And this is our private land that we've paid for, and we shouldn't be in any way apologetic about our right to privacy on it.
Frank