Where Can My Robot Go?

Where do robots find what pages are on a website?

Hint:

What does disallow tell a robot?

Solution:

User-agent: *
Disallow: /70r3hnanldfspufdsoifnlds.html

Flag: CTFlearn{r0b0ts_4r3_th3_futur3}