I think you should not restrict it at all (which means: set the limit at a very high level). It is difficult, if not impossible, to determine how long the fields should but they can not be too short, that's for sure. Names, cities and streets can be very long. Just check them to get a perspective:
Longest place names in the world:
Longest street names in the world:
Hey, my country wins! 🙂
For the longest surname check:
There is a world record on it!
The argument of not letting too many users stick is important to a certain extent, but I do not really think that it will cause more problems than fields too short to enter data.