Meta has paused its new business terms that would have let it crawl websites linked to Facebook Pages and ad accounts. The Meta scraping terms were due to start on November 20, but Meta now says its existing terms stay in place. Nothing changes for your site today, though a two-minute robots.txt check is still smart.
Here is what happened, who fought it, and what you can do right now.
These are the facts worth knowing first:
- Status: paused indefinitely, with no new start date, per Nieman Lab.
- Original date: November 20, 2026, after an earlier October 30 target.
- Who it covers: any business using Meta products, even one that only runs a Facebook Page.
- Your move: review your robots.txt rules for Meta’s crawlers.
Pausing is not the same as cancelling, so the issue is not closed.
What Meta planned to change
Meta wanted to update four business documents: Commercial Terms, Product Catalog Terms, Self-Serve Ad Terms and Ad Creative Generative AI Terms. According to The Next Web, the new wording would let Meta use automated software to crawl pages linked to a business and pull updates from them.
Legal experts told Nieman Lab the crawled content could end up training Meta’s AI. Meta pushed back on that reading. A revision on October 16 removed the word “train” and limited use of scraped content to advertising and business products.
Meta also said the update mostly gathered crawling rules that already existed across several documents. Critics weren’t convinced, and they stayed skeptical even after the edits.
Why the Meta scraping terms worry more than news sites
News publishers led the criticism, since their articles are the obvious target. But the Commercial Terms apply broadly. A bakery with a Facebook Page and a website could be covered the same way as a national newspaper.
That is the part small site owners miss. You agree to Meta’s business terms to run ads or a Page, and the terms decide what Meta may do with the site you link there.
If you already follow Meta’s other privacy changes, our guide to Ray-Ban Meta privacy settings covers a different corner of the same company’s data habits.
Who pushed back, and where it stands
The pause did not come out of nowhere. Regulators, courts and lawmakers all got involved, as the table below shows.
| Who | What they did |
|---|---|
| Norway | A court temporarily blocked the terms after a lawsuit from the country’s media association. |
| Brazil | Media groups sued, and a court issued a preliminary injunction. |
| UK CMA | Opened a formal investigation into whether Meta imposed exploitative terms on publishers and advertisers. |
| European Commission | Is assessing a complaint from publishers. |
| US senators | Adam Schiff and Marsha Blackburn wrote to Mark Zuckerberg asking him to withdraw the terms. |
Meta’s statement was short: “We’re pausing the changes as we continue to engage with stakeholders and take our time to get this right.” The injunctions and the UK investigation are still relevant.
How to block Meta crawlers in robots.txt
The simplest control you have is robots.txt, a plain text file that tells bots which pages they may visit. Meta lists its crawlers on its developer documentation page, and each one does something different.
| Crawler | What it does | Follows robots.txt? |
|---|---|---|
| meta-externalagent | Crawls to train foundation AI models or index content | Yes |
| meta-webindexer | Improves Meta AI search results and helps it cite your pages | Yes |
| meta-externalads | Crawls for advertising and business products | Yes |
| facebookexternalhit | Builds link previews on Facebook, Instagram and Messenger | Usually, but it may bypass for security checks |
| meta-externalfetcher | Fetches a link when a user asks, for agentic AI features | No, it may bypass robots.txt |
To block the AI training crawler, add these two lines to your robots.txt file:
User-agent: meta-externalagent
Disallow: /Changes can take up to 24 hours to apply, because crawlers cache the file. Be careful with facebookexternalhit: blocking it could break the rich previews people see when they share your links.
Also note that robots.txt is a request, not a lock. The user-initiated fetcher is documented as able to ignore it, so sensitive pages need real access controls.
What happens next
Meta has not said when the changes will return or whether the crawling rules will be revised. It earlier promised a full consultation with news industry leaders before going ahead, and a spokesperson says another update will come when there is more to report.
Watch for a new effective date in Meta’s Business Help Center. Any new date should show up there first. We covered Meta’s other recent AI push in Meta Muse avatar video calls.
Frequently Asked Questions
Is Meta scraping my website right now?
Meta already runs the crawlers listed in its documentation, and its existing terms remain in effect. The paused update would have added broader wording for pages linked to business accounts.
Do the terms apply if I only have a Facebook Page?
Per The Next Web, the Commercial Terms cover any business using Meta products, including one that only runs a Page. The paused changes would have reached those users too.
Will blocking meta-externalagent hurt my Facebook reach?
Meta describes it as a crawler for AI training and content indexing, separate from the link-preview crawler. Blocking it should not change how your shared links look, but check your analytics afterward.
Did Meta say it would train AI on scraped sites?
Not in the revised text. Meta removed the word “train” on October 16, but legal experts quoted by Nieman Lab said the terms still allowed broad scraping.
When will the terms come back?
Nobody knows. Meta has set no new date and says it is still consulting stakeholders.
My take: treat this pause as a free warning. If your content is your business, decide today which Meta crawlers you want, add the rules, and move on. If Meta brings the terms back, you will already be covered, and you will have lost nothing in the meantime.
Leave a Reply