---
title: "Transcript: Your Agent's Biggest Lie: \"I Searched the Web\" — Rafael Levi, Bright Data"
category: "transcripts"
videoId: "btxGmN8RvNU"
sourceLabels: ["YouTube transcript", "Cached transcript markdown"]
wordCount: "2716"
---

# Transcript: Your Agent's Biggest Lie: "I Searched the Web" — Rafael Levi, Bright Data

## Source Video
- [YouTube](https://www.youtube.com/watch?v=btxGmN8RvNU)

## Local Cache
- `raw/sources/youtube-transcripts/btxGmN8RvNU.txt`
- 2,716 words

## Transcript

Okay, so let's just work with it like this. It's a small room. Hi everybody. Welcome. My name is Rafael. I represent Bright Data. Bright Data is basically web access platform to help agents or anybody collect public data on scale. And I'm here to talk about the LLMs misleading people and all the time convincing them, "Hey, I did a search. Hey, I did this." While it didn't. Why? Because LLMs are programmed to please people, please users. So they make enough things and this is the biggest issue right now I'm seeing with LLMs. And I'm building applications all the time. I would rather LLM tell me, "No, I can't." But it never does.

It always tries to make things up. So currently the web is actually fighting the robots and automations. And it's been something that going on for years. Everybody knows CAPTCHAs. The first CAPTCHAs showed up year 10 like, you know, decade ago. And it just keeps growing. And now we have AI blocking AI and there's a whole world going on. And the web access is actually not as simple as it looks. So they're getting CAPTCHAs and they don't actually report CAPTCHAs. So it tries to go and find the data different way. Sometimes it goes into training data. And this is where the worst thing is.

When it uses training data and tells you that this is the current situation. Training data is from 2024. We're in 2026 and it doesn't add up, right? So these are some of the new things that I was just literally checking out, right? So Cloudflare blocks AI crawling for about 20% of the web, right? So 20% of the web is literally not accessible by AI by default fetch that's built into it. Cloudflare also right now released an AI labyrinth to actually trap bots and mislead them and provide to them fake data. So, then your results are getting even worse. Okay? So, the invisible failure group, right? There's no error, no warning, just the wrong answer, right?

So, the agent sends a request, it gets a capture, uh even an empty page, it doesn't tell you, "Hey, I just got an empty page." It will try to make up something, and this is where the most of hallucinations come from. The need to please and lack of data. So, it literally makes things up. I've seen it literally make numbers up, provide fake citations. You click on the citation, it's a 404, the page doesn't exist, and I'm sure all of you have uh I've seen that happening recently. I mean, literally like 60% of citations on ChatGPT is not working. How many of you have tried to purchase a product?

Hey, find me this product online, I want to buy it, and give me a link to the product. You click on the link to the product, and there's no product. Like, so what is this product that you're talking about? The URL doesn't exist, the product doesn't exist. So, where do I buy this product for the 50 bucks? Doesn't exist. Um so, what I want to show you, and I'm going to show you in code. How many of you are familiar with coding like um VS code? No? Okay, perfect. So, nobody's going to get lost if I'm going to switch to that.

So, I'm going to do a demo basically with MCP and without the Bright Data MCP, and we're going to compare what is going on there. So, first I'm going to I I I want to show you that I have exactly identical prompts for both of the uh scripts, right? So, without MCP and with MCP. So, the the I I give it five tasks. Uh Property, rightmove.co. Let's go check out some of the properties, LinkedIn, Instagram, Amazon, and TikTok. These are the five basic uh sites that I wanted to access. They are very heavy on anti-bot systems, and I want to show you the difference.

So, first I'm going to run it without an MCP, And I'm going to literally to let the AI talk for itself. And GPT-5 is a bit slow, so it's going to take some time. But basically, what we're trying to do is we're trying to access this URL. It's some local properties that I did literally half an hour ago. Um LinkedIn company in Israel, let's check it out. Instagram account, uh some Amazon product, and some TikTok. This is not limited to just these five websites. It's just something that I picked that is I know for sure will not work without MCP. So, as you can see, without MCP, I don't have live web data access.

Uh doesn't have any browsing tools, right? So, this is with tools not available just by default out of the box uh GPT-5. I mean, it's a strong but it's a strong LLM. And zero success, five failed. Same exact thing, exactly the same prompt, I'm running with our MCP. Uh our MCP has 66 tools. While it's running, I just want to go off some of the tools that it has. Search engine. Search engine is basically the LLM is able to do Google search, Bing search, DuckDuckGo search. Real searches, not just like, you know, search the the web that it's doing in the background. Um it has It also has scrape as a markdown.

That's a very strong one. Basically, it can send a curl to any URL and get just the markdown without HTML tags. So, you're not wasting tokens on parsing the HTML. Uh search engine batch. If you want to do, like, you know, like 100 keywords, you can literally can literally send 100 keywords and get 100 keyword batch results. Like, so the scaling is also huge. Discover. It also has pre-built APIs for many websites as you can see. And of course, it has a scraping browser infrastructure. So, it's a remote browser that your LLM can open and navigate. The remote browser solves capture by itself. It has a unique fingerprint.

So, it can open 100 browsers, navigate the same website without getting blocked. So, the whole idea is that with our MCP, not only can it do a single sessions, it can do multiple sessions in parallel. And as you can see, let's see. We have success for the right move. We have success for the LinkedIn Misral. And Instagram also worked. Um Amazon product, it has the information for the product itself. And and then what I did is on the second part, I asked LLMs to compare the results from no MCP with MCP. So, that you don't take my word for it. Let's see what the ChatGPT will actually say. If it didn't get stuck.

It looks like it's stuck for some reason. >> May I ask a question? >> Of course, please. I I love questions. >> For social profiles, do you have like uh accounts that you run? >> No, we only work with public data. >> Public data. >> Only publicly available data. Um collecting data behind login is not really legal. Why? Because when you sign up and create an account, you accept terms and conditions. When you accept terms and conditions, you need to really check it if it says, "Can you scrape?

Can you Do you allow Do they allow robots to access the website?" And that's why you Maybe some of you heard there's a lot of lawsuits going on. LinkedIn suing these people. Everybody's suing them. Amazon suing, yes. >> But I think even um for example, LinkedIn and Instagram, I think they don't even show you really any public data even if you're on the >> Well, there's plenty of public data for for LinkedIn, of course. If you take for example, I will take this URL. I might get blocked. I don't know how's the local IP, but um and I open an incognito window, right? There you go.

So, this is a public data that can be collected. Company, it's the same thing. It's the same thing. The only thing is they're very critical, right? So, if you are using, let's say, a Wi-Fi IP of a big event like this, it probably will block you because it's a data center IP, it's low quality IP. From home, you can probably access maybe 5, 10 profiles, but eventually it will also ask you to log in. Uh so, we only deal with public data. So, when you guys are using us, you are safe in the sense of nobody's going to come knocking on your doors to sue you, if that makes sense. No.

We Again, we don't deal with data behind login. We consider it to be illegal, so we don't deal with accepting terms and conditions and data behind login. Only publicly available. We have a whole data set. So, if you guys don't want to do live data and you don't care if it's a few months old, we have data sets literally that you can just filter by, let's say, that you're looking If we're talking about LinkedIn people, you're looking for AI engineers in a certain area, you can filter it and just get the data set right away. So, and and your agent will actually have access to that.

So, your agent can filter the data set and get you the data if you want to, right? So, it has access to all the tools. So, here's basically head-to-head comparison by the LLM itself. Without MCP failed, no live web access. With if listed, failed, successful, failed, successful. So, that's basically it. Uh antibot bypass, CAPTCHA solving, right? So, again, our system automatically solves CAPTCHA, so if your bot navigates to a website with a browser and has a CAPTCHA, our our browser has built-in capture solving solution so it will automatically solve the capture and your bot can continue on browsing without getting blocked. How much time I got? I don't know.

Um So just to kind of summarize it, right? This is the biggest hallucinations that you guys see. The agent gets blocked, it needs to please you and it makes things up. And um fake content also, right? So now uh if you want to Google what is Cloudflare Cloudflare AI Labyrinth it's basically a system once it detects a bot it's not doesn't block it. It literally feeds it fake data. So bigger hallucinations. Right? And um the easiest fix for this is just to make sure that your agent doesn't get blocked. And it's as easy as to implement our MCP. Uh our MCP has a free tier of 5,000 requests.

So if you guys want to try it out, um this is you can connect with me on LinkedIn if you want to. Or is this the Hold on a second. Is this the sign up for the MCP? I'm mostly in these QR codes. One second. Yeah, internet. Yes, please. >> Uh how does it detect if if there's Cloudflare Labyrinth or not? >> Um so the way we approach it is that we make your agent look like a human being. Literally like uh there's mouse movement pre-recorded, there's typing when it types it's like it's uh mimics a real human behavior. So uh the Cloudflare literally just doesn't even ask are you a robot or not, right?

So this is our our approach. Instead of trying to understand how they detect we make the agent look as human as possible so that it doesn't trigger the actual blockage. Misleading data is one of the toughest things that you can actually encounter. A lot of websites right now in Asia is doing that, right? Hotels, they're literally providing different prices. You go check out on your phone, you get one price. You can check from your computer, you get a different price. You can ask for a proxy, you get a third price. Which one is correct? It's really hard to tell.

When it comes into the domain of misleading, uh the best bet is to make sure that you your agent looks like a human, and then hope for the best. That That's basically what the approach is right now. Um AI levers was literally released a few like a month ago. Uh I don't have much of statistics on what is like, you know, exactly how it's working. All I know is that it didn't really affect us. We don't see any kind of change in data, and we're collecting petabytes of data on a daily like We have so many customers always scraping. We're caching data, so we're always comparing the results.

We don't see any degradation in the results. Uh so, I think we're doing a good job in that case. Uh this is a QR code that you guys can sign up for. Uh we have a GitHub page as well. Uh GitHub brightdata.com. Oh, just GitHub brightdata. And another thing that I would recommend for you guys to check out is the skills page, right? So, what we did is we created skills. And I'm going to have another session in a couple of hours.

If you're interested in seeing what the skills does, is basically you can take any agent, tell it to go here, and this page will teach it on how to build a scraper, or how to build a pipeline that will collect you the data. It has all the information it needs, all the APIs. And uh if you will come back to my next session, which is at 1:00, I think. Something like that. I'm going to literally demonstrate how it builds the pipeline. Literally in front of you, I'm going to tell it, "Hey, listen, let's build a Walmart collector for ABC." And it's going to build it, and it's going to scrape it.

And instead of parsing each individual HTML, it's going to build a parser, and it saves about 99% of the tokens. Because I see a lot of people is like, "Hey, I need to parse 10,000 pages, but it's so token heavy." Don't parse with LLM. LLM builds a parser and then the script runs it. But that's the next session. Any questions? >> Uh question on the performance. Uh I saw that your MCB exposes 69 tools, which means if I if I need to do the same surgical ability for my for my agent do I need to load all the 69 tools >> of course filter it.

You I just showed 69 tools because just to show it. Uh if I need just to scrape markdown and search, I would just literally load two tools. Uh otherwise you're flooding contacts with irrelevant data, of course not. I'm sorry, I didn't hear the beginning again. >> Uh the the experiment beginning >> Yes. Um so we didn't do any searches. Um I literally told it, "Hey, go to this URL, see if you can load it." Right? So that I didn't use the search uh in this demo. >> Sure. >> Uh but of course again even with our MCB they can actually do a Google search. >> Yeah.

>> And that's the one of the biggest benefits is because we're used to Google results. So by default when you're asking LLM, you expect it to do a Google search, but it doesn't. So the results with the MCB much much better and I I recommend sign up, try it out, it's free. See the results, compare what you guys get. >> Is that for 5,000 requests per day or >> Per month. Per month. Which is, you know, like for an MVP for a little experiment is more than enough.

Uh we also have pay as you go, so if you do need a little more it's it's nothing like, you know, it's Yeah, yes, but for a is perfect. I always I do a lot of hackathons and I'm always recommending, "Hey, listen, set up an MVP and tell your agent to go build whatever you need. It does a much better job than without."