Search for a command to run...
Reflections on trust, synthetic media, and why the real challenge isn't the technology but it's what happens when we stop knowing what to believe.
No comments yet. Be the first to comment.
This week I attended "Retrieval in the Age of Agents", a meetup organized by Qdrant and hosted at the Merantix AI Campus in Berlin. The event brought together representatives from Qdrant, Haystack (de
Vibe Coding vs AI Assissted Development
Disappointment A few weeks ago, I was in a fireside chat with Bob van Luijt, CEO & co-founder of Weaviate, where someone asked him about Knowledge Graphs. He ended his answer by saying, “I think that knowledge graphs are gonna cease to exist.”(You ca...
In this blog post I want to just simply gather any useful resource I found and updating it during my learning journey. Development LangChain What is LangChain? (YouTube video) (8mins) MCP MCP: Build Rich-Context AI Apps with Anthropic Knowledg...
TL;DR
Deepfakes are not just fake videos. They are part of a broader challenge around how Generative AI is reshaping trust both in the content we share and the information we consume.
The real risk isn't only technical. As AI-generated content becomes indistinguishable from reality, journalism, elections, humanitarian work, businesses, and everyday communication all depend on new ways of establishing trust.
Technology alone won't solve this. Watermarking, detection, and metadata help, but lasting solutions require collaboration between AI companies, policymakers, researchers, civil society, businesses, and users.
AI also influences what we believe. Beyond creating fake media, conversational AI can subtly reinforce biases and shape opinions at massive scale, making AI literacy increasingly important.
My biggest takeaway: The future of AI governance isn't just about regulating models but it's about building an ecosystem where trust, transparency, and collaboration evolve as quickly as the technology itself.
"Seeing is believing" has been one of humanity's default assumptions for centuries. Today, that assumption is becoming increasingly unreliable.
Last night I attended a fascinating panel discussion on deepfakes, bringing together experts from humanitarian organizations, AI ethics, industry, and European policy. While the conversation focused on synthetic media, what stayed with me afterward wasn't a technical question but this:
When did we stop talking about deepfakes and start talking about Generative AI?
The two terms are not interchangeable, but I wonder if our excitement around Generative AI has made us forget that many of the concerns we once associated with "deepfakes" are now embedded in the broader AI ecosystem.
Technically, a deepfake is a specific application of generative AI: using AI to create or manipulate images, audio, or video to make something appear authentic when it isn't.
Generative AI is much broader. It writes text, generates code, creates music, produces images, and increasingly acts as an assistant, tutor, or companion.
But I think there are two ways to think about "deepfakes."
The first is the one most people associate with the term: content generated to deceive others. A fake political speech, a manipulated video, non-consensual intimate images, or fraudulent voice cloning.
The second is less discussed.
As we increasingly interact with AI systems every day, we also consume information generated by them. These systems often communicate with remarkable confidence, even when they're wrong. The content isn't necessarily malicious, but it can still create a convincing version of reality that doesn't exist.
In that sense, there is another kind of deep fake: information that feels trustworthy simply because an AI generated or presented it convincingly.
This isn't about fake videos or manipulated images. It's about the subtle way conversational AI can shape what we believe over time.
Research on human–AI interaction suggests that people naturally anthropomorphize AI systems, assigning them human characteristics such as competence, expertise, or even empathy (see Blut et al., 2021; Cohn et al., 2024). The more human-like an AI appears through its language or voice, the more likely people are to trust its answers, sometimes beyond what its actual reliability deserves.
That becomes more concerning when these systems reflect biases in their training data or conversational behavior. Unlike a traditional search engine that presents multiple sources, a chatbot often synthesizes a single, coherent answer. Even if that answer is only subtly biased, repeated interactions can gradually reinforce certain perspectives while making alternative viewpoints less visible.
The implications become even greater for the next generation. Many children growing up today will not experience AI as a novel technology. They will experience it as a constant companion: helping them learn, answering their questions, assisting with homework, and eventually influencing how they work and make decisions. For them, AI may become a primary interface to knowledge rather than simply another tool and for some even their best friend who gives them the feeling that understands them "better than anybody else".
That raises an important question: what happens when the first source of information for millions of people is a system that confidently summarizes the world instead of exposing them to multiple perspectives? If its biases—however small—are repeated billions of times across years of interactions, they could shape not only individual opinions but also how future generations develop critical thinking, curiosity, and their understanding of truth itself.
This isn't necessarily propaganda, nor is it evidence of intentional manipulation. But at the scale of hundreds of millions of daily interactions, even small systematic biases or tendencies toward overconfidence could influence how societies form opinions, whom they trust, and how public discourse evolves.
Recent research from the University of Washington (see Fisher et al., 2025) found that after only a few interactions, people tended to shift their opinions in the direction of politically biased chatbot responses. Interestingly, participants with greater AI literacy were significantly less susceptible to this effect, suggesting that education may become one of the most effective safeguards against AI-driven influence.
This second angle wasn't the focus of the panel, but it kept coming back to my mind throughout the discussion. The challenge isn't only what we share with others; it's also what we gradually learn to trust ourselves.
One recurring theme throughout the discussion was that deepfakes are no longer just a technical problem.
They're becoming an infrastructure problem for society.
When synthetic media can convincingly imitate reality, every institution built on trust starts to feel the impact:
journalism
elections
humanitarian work
education
businesses
financial fraud
legal systems
even everyday family communication
One speaker shared a quote often attributed to wartime reporting:
"The first casualty of war is truth."
In a world of synthetic media, truth isn't only threatened during conflicts. It becomes harder to establish in everyday life.
Imagine trying to verify a ceasefire announcement, a humanitarian message, an emergency request from a family member, or even whether a government website is genuine. Once people lose confidence in what they see and hear, the cost extends far beyond individual scams.
Trust itself becomes scarce.
One statistic from the discussion particularly stood out.
Researchers from the Oxford Internet Institute identified nearly 35,000 publicly downloadable deepfake model variants, downloaded almost 15 million times since late 2022. Many of these models are specifically fine-tuned to generate images of identifiable people, with 96% targeting women.
The study also showed how accessible this has become: some models can be created using as few as 20 images, consumer-grade hardware, and around 15 minutes of processing time.
The concern is no longer whether this technology exists.
It is already widely available.
When discussions around deepfakes begin, they often end with the same proposed solutions:
Watermarks.
Detection models.
Content authenticity.
Metadata.
These are all important but none of them is sufficient on its own.
Watermarks can be removed.
Detection models improve, but so do generation models.
Authentication standards help, but only when adopted consistently.
This isn't a race that has a single technical finish line.
Instead, detecting manipulated content increasingly requires combining technical signals with contextual information, provenance, metadata, and trusted sources.
The conversation also highlighted why regulation struggles to keep pace.
Technology evolves faster than legislation.
By the time policymakers understand one generation of AI capabilities, the next generation has already arrived.
That doesn't mean governments are inactive. As the EU AI Act continues to come into force, transparency has become one of its central principles. More recently, the European Commission published the Code of Practice on Transparency of AI-Generated Content, providing practical guidance on how providers and deployers can label AI-generated and manipulated content and comply with the Act's transparency obligations. While the Code is voluntary, the underlying transparency requirements are legal obligations under Article 50 of the AI Act.
But regulation alone cannot solve the problem.
Some harms already fall into legal gray areas. Others span multiple jurisdictions. Meanwhile, the companies developing these systems operate globally while governments legislate nationally or regionally.
Another interesting observation was that regulation itself has become increasingly geopolitical. The discussion is no longer only about what is ethically desirable, but also about competitiveness, strategic interests, and global influence.
That makes the challenge considerably harder.
Perhaps my biggest takeaway wasn't about AI at all.
It was about collaboration.
Throughout the discussion, it became clear that every stakeholder depends on the others:
AI companies build the models.
Researchers study the risks.
Civil society understands the human impact.
Industry decides which safeguards to adopt.
Governments create legal frameworks.
Buyers determine which technologies succeed in the market.
Humanitarian organizations experience the consequences where trust is already fragile.
None of these groups can solve the problem independently:
Technical experts need policymakers who understand implementation realities.
Regulators need researchers who can explain emerging risks and what tools can be used to mitigate them.
Companies need practical standards instead of vague expectations.
Users need clearer ways to understand when they are interacting with synthetic content.
The conversation shouldn't happen in separate rooms.
It needs to happen around the same table.
Leaving the event, I found myself returning to the question that started all this.
Has "deepfake" simply become the forgotten name of Generative AI?
Not exactly.
Deepfakes are one application of Generative AI, and an important distinction remains. Most generative AI tools are built for legitimate and often valuable purposes.
But the discussion reminded me that the underlying issue has never really been about the technology itself.
It's about trust.
How do we preserve trust when creating convincing synthetic content becomes almost effortless?
How do we help people distinguish authenticity from fabrication?
And perhaps the more uncomfortable question:
Those are not questions that engineers, lawmakers, or companies can answer alone.
They are questions that society will have to answer together.
This post was inspired by a panel discussion hosted by Merantix Momentum featuring:
Philippe Stoll (Swiss Red Cross / Red Cross Red Crescent Hub on Harmful Information)
Alessandro Polidoro (European Commission)
The article reflects my own interpretation and takeaways from the discussion plus recent research on deepfakes, rather than a transcript of the panel.
Oxford Internet Institute. Dramatic Rise in Publicly Downloadable Deepfake Image Generators.
European Commission. Code of Practice on Transparency of AI-Generated Content.
Blut, M., et al. (2021). Understanding anthropomorphism in service provision: A meta-analysis of physical robots, chatbots, and other AI.
Cohn, M., et al. (2024). Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Models.
Fisher, J., et al. (2025). Biased LLMs Can Influence Political Decision-Making.
