Run your first crawl
Let Meelu read your website so every desk has something to work with.
A crawl is when Meelu visits your website and reads its pages one by one, the way a search engine does. Until the first crawl finishes, most desks have nothing to show. The crawl runs on Meelu's server, so you can close the tab while it works, and it uses no credits. After the first crawl, Meelu reads your site again every day on its own.
Start the crawl
Open Technical SEO
Click Technical SEO in the sidebar. It opens on the Health view. Until the first crawl, it says This website has not been read yet.
Click Crawl website
The button is at the top right of the Health view, and also in the middle of the empty screen.
Follow the progress
A progress bar shows what the crawl is doing. You can use other desks while it runs. A small site takes a few minutes. A site with thousands of pages takes longer.
Look at what it found
When the crawl ends, the Health view shows six numbers: Sitemap coverage, Broken, Not in sitemap, Orphans, Unreachable and 4+ clicks deep. The Pages, Internal links and Technical SEO sections fill in too.
- 1Crawl again reads the whole site again
- 2Ask Meelu opens ready-made questions for this desk
- 3Pages lists every page the crawl read
- 4Internal links shows link and pillar page suggestions
- 5Technical SEO lists technical problems
What the progress bar says
| Message | What is happening |
|---|---|
| Reading the sitemap and robots.txt | Meelu reads your sitemap (the file that lists your pages for search engines) and your robots.txt (the file that tells crawlers what they may read). |
| Reading the pages the sitemap lists | Meelu reads every page your sitemap lists. |
| Following links the sitemap did not list | Meelu follows links on your pages to find pages the sitemap missed. |
| Stopping — the pages already read are kept | You clicked Stop crawl. Nothing already read is lost. |
If something goes wrong
After the first crawl
Meelu reads your site again every day. Click Crawl again if you changed your site, for example after you publish a page or fix broken links, and do not want to wait.
For everything about crawling, including limits and pages Meelu skips, see Crawling your site.
What each button does
| Button | What it does |
|---|---|
| Crawl website | Starts the first crawl of this website. |
| Crawl again | Starts a new crawl of a website that has been crawled before. |
| Stop crawl | Stops the crawl and keeps the pages already read. |
| Use this sitemap | Crawls using the sitemap address you pasted, and remembers it. |