importance of robots.txt

i want to know the importance of robots.txt for a website