Over 90% of video processing workflows online and offline involve FFmpeg in some form.
#496 – FFmpeg: The Incredible Technology Behind Video on the Internet
The entire global video internet — YouTube, Netflix, Chrome, Discord — runs on FFmpeg, maintained by roughly 10–15 volunteers who have never been paid a meaningful salary for it.
Lex Fridman Podcast
#496 – FFmpeg: The Incredible Technology Behind Video on the Internet
The entire global video internet — YouTube, Netflix, Chrome, Discord — runs on FFmpeg, maintained by roughly 10–15 volunteers who have never been paid a meaningful salary for it.
TL;DR
FFmpeg and VLC — the open-source software powering nearly all video on the internet — are built and maintained by a tiny band of volunteers. Jean-Baptiste Kempf and Kieran Kunhya explain how video compression works, why hand-written assembly outperforms C by up to 62x [1] — Jean-Baptiste Kempf "When the Alliance for Open Media said AV1 was too complex for software decoding, VideoLAN proved them wrong with dav1d: 240,000 lines of ha…" 2:08:50 , how JB repeatedly turned down tens of millions of dollars to keep VLC ad-free [2] — Jean-Baptiste Kempf "JB didn't turn down advertising money because he hates money. He turned it down because every offer came from shady ad companies, and accep…" 1:00:59 , the Google security-researcher drama that boosted FFmpeg donations [3] — Kieran Kunhya "Google used AI to mass-generate security vulnerability reports on FFmpeg — including a 'critical' bug in an obscure 1993 game codec — and w…" 1:15:17 , and why the dav1d AV1 decoder's 240,000 lines of handwritten assembly represent a modern engineering masterpiece [4] — Kieran Kunhya "Reverse engineering a codec means opening a binary with no documentation, stepping through millions of CPU instructions in a debugger, dump…" 2:00:00 . The single most useful takeaway: the entire digital video world runs on volunteer passion projects — support them.
Jean-Baptiste Kempf (president of VideoLAN, lead VLC developer) and Kieran Kunhya (codec engineer, FFmpeg contributor) join Lex Fridman to discuss the invisible technology powering all video on the internet: FFmpeg and VLC. Topics span video codecs, containers, open source philosophy, assembly programming, reverse engineering, the Google security drama, turning down millions to keep VLC ad-free, and the future of multimedia.
-
Lex introduces FFmpeg and VLC, praises the open-source volunteer engineers behind them, and gives a rapid sponsor roll call before the full ad reads begin.
-
Extended sponsor reads for Larridin, Blitzy, BetterHelp, Fin, and LMNT, with Lex weaving in personal reflections on AI, mental health, and electrolytes.
-
VLC's legendary resilience: VHS capture cards, DVD Audio, Star Wars game codecs, competition files with per-frame resolution changes, and the infamous cone logo origin. [1] — Jean-Baptiste Kempf "25% of the website traffic that comes to our main website is 'Kun Player,' right? So many people don't know VLC, they know the Kun Player." 13:12
-
Step-by-step breakdown of decoding: demuxing, entropy coding, intra-prediction, DCT, GPU vs. software paths, and why 45% of files need software fallback. [1] — Kieran Kunhya "Everything we've just said in the past couple of minutes, every sentence is someone's lifetime's work. There are books about every sentence." 24:03
-
Containers (MP4, MKV, MOV) vs. codecs (H.264, AAC); why VLC ignores file extensions; the 100–1000x compression target; YUV color space and human perception. [1] — Jean-Baptiste Kempf "Video compression isn't just removing data — it's degrading reality in ways humans don't notice. Codecs exploit the fact that your eyes car…" 19:10
-
FFmpeg as a language-like toolbox; its democratizing role in podcasting and YouTube; open-source licensing (MIT, GPL, LGPL); and why changing a license requires contacting every contributor. [1] — Jean-Baptiste Kempf "FFmpeg is not just a command-line tool — it's a complete multimedia processing universe. It handles codecs, containers, filters, and stream…" 35:20
-
The legend of Linus Torvalds, Git built in two weeks, the meritocratic harshness of code review culture, and why a core team of 10–15 must maintain code for thousands of contributors.
-
The full story behind JB's Reddit-famous refusal of tens of millions in ad deals, the VideoLAN origin at École Centrale Paris, and why accepting would have betrayed hundreds of volunteers. [1] — Jean-Baptiste Kempf "JB didn't turn down advertising money because he hates money. He turned it down because every offer came from shady ad companies, and accep…" 1:00:59
-
Google's AI-generated bug reports on obscure codecs, the 'denial of service by CVE' critique, Microsoft Teams demanding free urgent support, and how public X posts forced accountability. [1] — Kieran Kunhya "Google used AI to mass-generate security vulnerability reports on FFmpeg — including a 'critical' bug in an obscure 1993 game codec — and w…" 1:15:17
-
Passion for multimedia, the school of programming that beats any university, pride of shipping code used by billions, and teenagers writing production assembly. Andrew Kelly (Zig) started as an FFmpeg developer.
-
The interdependence of VLC, FFmpeg, and x264 explained as a binary star system — VLC is to FFmpeg as Android is to Linux; x264 powers 80%+ of FFmpeg encoding pipelines.
-
Fabrice Bellard creates the concept; Michael Niedermayer's 2000s era of DivX/Xvid support; the H.264 maturity era; native decoders replacing bloated proprietary players like RealPlayer.
-
Kostya's GoToMeeting reverse engineering, stepping through millions of CPU instructions, dumping memory from VMs, needing bit-exact output — and why a 1MB blob takes a month. [1] — Kieran Kunhya "Reverse engineering a codec means opening a binary with no documentation, stepping through millions of CPU instructions in a debugger, dump…" 2:00:00
-
The FATE automated testing system covers hundreds of compiler/OS/architecture combinations on volunteer hardware, catching silent miscompilations that could cascade into video glitches.
-
Why SIMD assembly beats C by 62x; dav1d's 240,000 lines of handwritten assembly; Henrik Gramner and Martin Storsjö as assembly wizards; the Moore's Law plateau making optimization essential. [1] — Kieran Kunhya "Compilers cannot auto-vectorize anywhere close to what an expert human can do by hand. The dav1d AV1 decoder proves it: 80% assembly, 62x f…" 1:50:40 [2] — Jean-Baptiste Kempf "When the Alliance for Open Media said AV1 was too complex for software decoding, VideoLAN proved them wrong with dav1d: 240,000 lines of ha…" 2:08:50
-
JB and Kieran debate Rust: valuable for new projects and memory safety, but ill-suited for rewriting existing assembly-heavy codebases, and hand-written assembly breaks any Rust security model anyway.
-
The 2011 governance split, how libav's work was absorbed back into FFmpeg, how forks are a healthy part of open-source evolution, and why FFmpeg emerged stronger.
-
AI slop bug reports, the XZ backdoor caused by deliberate burnout of a solo maintainer, death threats JB received over dropping PowerPC support, and the mental health toll on critical infrastructure maintainers. [1] — Jean-Baptiste Kempf "The XZ backdoor happened because a single overwhelmed maintainer was systematically harassed at night by two attackers posing as contributo…" 2:48:17
-
How anime fans rejected PSNR metrics and invented psychovisual rate distortion; adaptive quantization; the Parkjoy sample as the benchmark that sorted encoders; Blu-ray adoption of x264. [1] — Kieran Kunhya "For decades, video codec engineers optimized for PSNR — a purely mathematical metric that made video look blurry. Hobbyist anime fans ignor…" 2:56:04
-
I-frames, P-frames, B-frames, intra-refresh, group-of-pictures, AV1 vs. H.264 codec tool collections, ProRes intra-only for editing, and why decoding order differs from display order.
-
WikiLeaks revealed the CIA distributed a fake VLC build with a DLL that exfiltrated documents during movie playback; Chinese hackers also used a signed VLC DLL; VLC was temporarily banned in India. [1] — Jean-Baptiste Kempf "At least two intelligence agencies approached VideoLAN requesting backdoors in VLC. The answer was an unambiguous no. VLC compiles on air-g…" 3:55:50
-
JB's Kyber startup targets 4ms glass-to-glass latency for robot teleoperation using QUIC, forward error correction, multi-stream clock synchronization, and 3D-printed demonstration rovers. [1] — Jean-Baptiste Kempf "Real-time robot control demands video latency measured in milliseconds, not seconds. Kyber encodes a single video frame in 4 milliseconds, …" 3:16:30
-
AV2 as 30% better than AV1; the HEVC patent disaster that cost hundreds of millions per year and drove creation of the Alliance for Open Media; dav2d as the planned AV2 decoder. [1] — Jean-Baptiste Kempf "350+ contributors contacted for relicense: To relicense VLC's core from GPL to LGPL, Jean-Baptiste had to personally track down and contact…" 52:24
-
Two intelligence agencies requested VLC backdoors and were refused; air-gapped build machines; double-signature verification; detected state-actor fake binary injection; the sandboxing architecture challenge. [1] — Kieran Kunhya "For decades, video codec engineers optimized for PSNR — a purely mathematical metric that made video look blurry. Hobbyist anime fans ignor…" 2:56:04
-
The archiving community led by Dave Rice uses FFmpeg and FFV1 lossless codec to preserve world multimedia heritage; the Domesday Book failure; C as the Latin of programming languages; the moral hazard of choosing what to archive. [1] — Kieran Kunhya "FFmpeg runs on Mars: FFmpeg is used by NASA's Mars 2020 rover to compress images, making it a multi-planetary open-source library." 3:18:55
-
VLC and FFmpeg expanding to volumetric video, XR streaming, haptic tracks, spatial audio, brainwave codecs; JB's philosophy on regret; closing tribute to open-source engineers from Linus Torvalds quote.
- Codec
- Short for coder-decoder; a program that compresses (encodes) and decompresses (decodes) digital audio or video data.
- Container / Mux
- A file format (e.g., MP4, MKV) that bundles multiple tracks — video, audio, subtitles — into one file; multiplexer = mux, demultiplexer = demux.
- SIMD
- Single Instruction Multiple Data; a class of CPU instructions that performs the same operation on multiple data elements simultaneously, crucial for fast video processing.
- I-frame / Keyframe
- A complete, independently decodable video frame, analogous to a JPEG image; all other frame types reference it.
- B-frame
- A bi-directionally predicted video frame that can reference both past and future frames, enabling higher compression at the cost of decoding complexity.
- YUV
- A color space separating luminance (Y) from chrominance (U, V); exploits human visual sensitivity to brightness over color to enable efficient compression.
- DCT (Discrete Cosine Transform)
- A mathematical transform that converts spatial pixel data into frequency-domain coefficients, allowing video codecs to discard perceptually insignificant detail.
- Entropy coding
- Lossless compression applied to the mathematical symbols in a codec bitstream; common methods include Huffman coding and arithmetic coding.
- FATE
- FFmpeg Automated Testing Environment; a volunteer-run continuous integration system testing FFmpeg across hundreds of compiler, OS, and architecture combinations.
- LGPL
- GNU Lesser General Public License; a 'weak copyleft' license allowing proprietary software to link against a library without open-sourcing the whole application.
- Copyleft
- A licensing principle requiring that derivative works be distributed under the same open-source license, ensuring modifications remain publicly available.
- Intra-prediction
- A technique where a video codec predicts pixel values from neighboring pixels within the same frame, exploiting spatial redundancy.
- AV1
- A royalty-free video codec developed by the Alliance for Open Media, offering roughly 40–60% better compression than H.264 at equivalent quality.
- Psychovisual rate distortion
- A codec optimization strategy that tunes compression decisions to match human visual perception rather than minimizing a purely mathematical error metric like PSNR.
- Forward error correction (FEC)
- A technique that deliberately over-transmits redundant data so the receiver can reconstruct lost packets without requesting a retransmission, reducing latency.
- QUIC
- A modern transport protocol built on UDP, designed for low latency and built-in encryption; used by Kyber for real-time video control streams.
- PSNR
- Peak Signal-to-Noise Ratio; a mathematical metric for image quality widely used in video coding research, often criticized for not correlating well with perceived visual quality.
- Demux / Demultiplexer
- The software component that reads a container file and separates it into its constituent tracks (video, audio, subtitles) for decoding.
- Meritocratic
- A system where advancement is based on demonstrated ability and output rather than seniority, status, or personal relationships; how FFmpeg and VLC evaluate code contributions.
- Bit-exact
- A guarantee that every compliant decoder produces identical output bits for a given input; standard from H.264 onwards but absent in MPEG-2, which caused interoperability problems.
Chapter 3 · 10:48
Weirdest things VLC opens
VLC's legendary resilience: VHS capture cards, DVD Audio, Star Wars game codecs, competition files with per-frame resolution changes, and the infamous cone logo origin. [1] — Jean-Baptiste Kempf "25% of the website traffic that comes to our main website is 'Kun Player,' right? So many people don't know VLC, they know the Kun Player." 13:12
Claims made here
VLC has been downloaded at least 6.5 billion times.
It is estimated that over 90% of online and offline video processing workflows involve FFmpeg in some form.
VLC has been downloaded at least 6.5 billion times, with the real number likely even higher due to counting difficulties.
Chapter 4 · 15:12
How video playback works
Step-by-step breakdown of decoding: demuxing, entropy coding, intra-prediction, DCT, GPU vs. software paths, and why 45% of files need software fallback. [1] — Kieran Kunhya "Everything we've just said in the past couple of minutes, every sentence is someone's lifetime's work. There are books about every sentence." 24:03
Claims made here
Approximately 45% of video files are not GPU-decodable and require a software fallback.
Video codecs compress raw video by 100x to 1000x compared to uncompressed formats, compared to about 10x for MP3 audio.
Each new generation of video codec delivers approximately 30% better compression at the same visual quality level.
Video compression isn't just removing data — it's degrading reality in ways humans don't notice. Codecs exploit the fact that your eyes care more about brightness than color, and that the next frame is usually almost identical to the last. The target is 1000x compression, achieved through mathematical transforms, spatial prediction, and temporal redundancy removal.
Video codecs compress raw video data by 100x to 1000x compared to uncompressed formats, far exceeding audio compression ratios.
Each new generation of video codec delivers roughly 30% better compression at the same quality level, at the cost of significantly more encoding compute.
Chapter 6 · 35:20
FFmpeg explained
FFmpeg as a language-like toolbox; its democratizing role in podcasting and YouTube; open-source licensing (MIT, GPL, LGPL); and why changing a license requires contacting every contributor. [1] — Jean-Baptiste Kempf "FFmpeg is not just a command-line tool — it's a complete multimedia processing universe. It handles codecs, containers, filters, and stream…" 35:20
Claims made here
Jean-Baptiste Kempf contacted more than 350 individual contributors to relicense VLC's core from GPL to LGPL.
FFmpeg is not just a command-line tool — it's a complete multimedia processing universe. It handles codecs, containers, filters, and streaming, and it's embedded inside YouTube, Netflix, Chrome, VLC, OBS, and virtually every smart TV. Your grandmother's home videos and Netflix's trillion-dollar infrastructure use the exact same technology stack.
Open source isn't just giving away software — it's giving you the cheesecake, the recipe, the instructions to build the oven, AND permission to modify and resell it. That openness is what allowed 2,000–3,000 contributors across the world to build the backbone of all internet video.
Since FFmpeg's creation, approximately 2,000 to 3,000 people have contributed code to the project over its lifetime.
To relicense VLC's core from GPL to LGPL, Jean-Baptiste had to personally track down and contact more than 350 individual contributors.
Chapter 7 · 56:20
Linus Torvalds
The legend of Linus Torvalds, Git built in two weeks, the meritocratic harshness of code review culture, and why a core team of 10–15 must maintain code for thousands of contributors.
The entire FFmpeg project is maintained by a core team of only 10 to 15 people, with around 6 to 8 at the core of VLC.
Chapter 8 · 1:00:59
Turning down millions to keep VLC ad-free
The full story behind JB's Reddit-famous refusal of tens of millions in ad deals, the VideoLAN origin at École Centrale Paris, and why accepting would have betrayed hundreds of volunteers. [1] — Jean-Baptiste Kempf "JB didn't turn down advertising money because he hates money. He turned it down because every offer came from shady ad companies, and accep…" 1:00:59
JB didn't turn down advertising money because he hates money. He turned it down because every offer came from shady ad companies, and accepting would have betrayed hundreds of people's volunteer labor. He could be a multimillionaire on a beach — he chose to keep VLC ad-free for billions of users instead.
Jean-Baptiste Kempf repeatedly turned down tens of millions of dollars in offers — mainly from shady ad companies — to keep VLC free and ad-free.
VLC started as a student hack at a French engineering school trying to solve token-ring latency for Doom. A satellite TV experiment led to one of the world's first video streaming projects. Without that 1994 campus network problem, there would be no VLC, no VideoLAN, no dav1d.
Chapter 9 · 1:15:17
FFmpeg & Google drama
Google's AI-generated bug reports on obscure codecs, the 'denial of service by CVE' critique, Microsoft Teams demanding free urgent support, and how public X posts forced accountability. [1] — Kieran Kunhya "Google used AI to mass-generate security vulnerability reports on FFmpeg — including a 'critical' bug in an obscure 1993 game codec — and w…" 1:15:17
Google used AI to mass-generate security vulnerability reports on FFmpeg — including a 'critical' bug in an obscure 1993 game codec — and went to the media before patches could be deployed. Volunteers faced a flood of high-severity reports they never had resources to fix. The result: FFmpeg pushed back publicly, donations rose, and Google eventually started sending patches.
A Microsoft Teams manager filed a bug on FFmpeg's public tracker, name-dropped Microsoft's brand to imply urgency, and when asked for a support contract, offered a one-time payment of a few thousand dollars. FFmpeg's response on X went viral, and the incident forced a reckoning about trillion-dollar companies freeloading on volunteer infrastructure.
Chapter 13 · 1:48:59
Reverse engineering codecs
Kostya's GoToMeeting reverse engineering, stepping through millions of CPU instructions, dumping memory from VMs, needing bit-exact output — and why a 1MB blob takes a month. [1] — Kieran Kunhya "Reverse engineering a codec means opening a binary with no documentation, stepping through millions of CPU instructions in a debugger, dump…" 2:00:00
Claims made here
Hand-written SIMD assembly for pixel format conversion at Open Broadcast Systems runs 62 times faster than equivalent C code.
AV1 delivers 40–60% less bandwidth consumption than H.264 at equivalent visual quality.
AV1 encoding requires approximately two orders of magnitude more CPU cycles than H.264 encoding.
VLC's latest version still supports Windows XP through Windows 11 and iOS 9 through iOS 26.
Hand-written SIMD assembly for pixel format conversion functions in Kieran's company runs 62 times faster than equivalent C code.
Compilers cannot auto-vectorize anywhere close to what an expert human can do by hand. The dav1d AV1 decoder proves it: 80% assembly, 62x faster than C on pixel conversion functions, running on 3 billion devices. Moore's Law is over — squeezing hardware through hand-crafted assembly is how you get more without buying more.
AV1 achieves 40–60% bandwidth reduction compared to H.264 at the same visual quality, enabling better streaming at lower bitrates.
Reverse engineering a codec means opening a binary with no documentation, stepping through millions of CPU instructions in a debugger, dumping memory from a virtual machine, and slowly inferring — frame by frame — what the compression algorithm is doing. A 1MB blob takes roughly a month. Kostya did 20–30MB blobs regularly, for fun.
VLC's latest version still supports iOS 9 through the current iOS 26, achieved through elaborate Frankenstein compiler configurations.
Chapter 14 · 2:02:14
FFmpeg testing
The FATE automated testing system covers hundreds of compiler/OS/architecture combinations on volunteer hardware, catching silent miscompilations that could cascade into video glitches.
FFmpeg's FATE testing system runs on volunteer hardware across hundreds of compiler, OS, and CPU variants — from Visual Studio to Apple Clang, from x86 to RISC-V to PowerPC. A single code change can silently miscompile on GCC 11 on macOS. The matrix of combinations isn't a table — it's a pivot table.
Chapter 15 · 2:06:21
Assembly code (handwritten)
Why SIMD assembly beats C by 62x; dav1d's 240,000 lines of handwritten assembly; Henrik Gramner and Martin Storsjö as assembly wizards; the Moore's Law plateau making optimization essential. [1] — Kieran Kunhya "Compilers cannot auto-vectorize anywhere close to what an expert human can do by hand. The dav1d AV1 decoder proves it: 80% assembly, 62x f…" 1:50:40 [2] — Jean-Baptiste Kempf "When the Alliance for Open Media said AV1 was too complex for software decoding, VideoLAN proved them wrong with dav1d: 240,000 lines of ha…" 2:08:50
Claims made here
The dav1d AV1 decoder contains approximately 240,000 lines of handwritten assembly code — more than double FFmpeg's total assembly across all codecs combined.
Approximately 30% of Netflix video content is now delivered in AV1, and approximately 50% of YouTube content is in AV1.
FFmpeg is likely running on 100 million to potentially 1 billion CPUs simultaneously at any given moment.
When the Alliance for Open Media said AV1 was too complex for software decoding, VideoLAN proved them wrong with dav1d: 240,000 lines of handwritten assembly, every cycle optimized, running on 3 billion devices. It enables 720p AV1 decoding on just one or two CPU cores — on hardware that was never supposed to handle it.
The dav1d AV1 decoder, a VideoLAN project, contains approximately 240,000 lines of handwritten assembly — more than double FFmpeg's total assembly across all codecs.
Approximately 30% of Netflix's video is now delivered using the AV1 codec, with YouTube at about 50%.
WikiLeaks' Vault 7 revealed that the CIA distributed a custom VLC build with a hidden DLL that read your documents folder and encrypted and exfiltrated the data — while you watched a movie for two hours and blamed your fans on the CPU load. VideoLAN's only defense was telling people to only download from videolan.org.
Chapter 16 · 2:40:40
Rust programming language
JB and Kieran debate Rust: valuable for new projects and memory safety, but ill-suited for rewriting existing assembly-heavy codebases, and hand-written assembly breaks any Rust security model anyway.
Chapter 18 · 2:48:17
Open source burnout
AI slop bug reports, the XZ backdoor caused by deliberate burnout of a solo maintainer, death threats JB received over dropping PowerPC support, and the mental health toll on critical infrastructure maintainers. [1] — Jean-Baptiste Kempf "The XZ backdoor happened because a single overwhelmed maintainer was systematically harassed at night by two attackers posing as contributo…" 2:48:17
The XZ backdoor happened because a single overwhelmed maintainer was systematically harassed at night by two attackers posing as contributors until he gave them commit access. AI-generated bug reports and corporate entitlement are now accelerating this crisis across all of open source. The entire internet runs on people like this.
Chapter 19 · 2:56:04
x264 and internet video
How anime fans rejected PSNR metrics and invented psychovisual rate distortion; adaptive quantization; the Parkjoy sample as the benchmark that sorted encoders; Blu-ray adoption of x264. [1] — Kieran Kunhya "For decades, video codec engineers optimized for PSNR — a purely mathematical metric that made video look blurry. Hobbyist anime fans ignor…" 2:56:04
For decades, video codec engineers optimized for PSNR — a purely mathematical metric that made video look blurry. Hobbyist anime fans ignored the sacred metrics and optimized by eye instead, inventing psychovisual rate-distortion tuning. x264 implemented their ideas and became the reference every codec since has been compared against.
Chapter 21 · 3:16:17
CIA and fake VLC
WikiLeaks revealed the CIA distributed a fake VLC build with a DLL that exfiltrated documents during movie playback; Chinese hackers also used a signed VLC DLL; VLC was temporarily banned in India. [1] — Jean-Baptiste Kempf "At least two intelligence agencies approached VideoLAN requesting backdoors in VLC. The answer was an unambiguous no. VLC compiles on air-g…" 3:55:50
Claims made here
FFmpeg is used by NASA's Mars 2020 rover to compress pictures sent back to Earth.
Real-time robot control demands video latency measured in milliseconds, not seconds. Kyber encodes a single video frame in 4 milliseconds, uses QUIC-based multi-stream transport with forward error correction, and handles clock drift across multi-camera robot setups to guarantee coherent training data for AI models. Current achievement: 7ms glass-to-glass. Target: 4ms (240 Hz equivalent).
FFmpeg is used by NASA's Mars 2020 rover to compress images, making it a multi-planetary open-source library.
Chapter 23 · 3:44:10
AV2 codec and video patents
AV2 as 30% better than AV1; the HEVC patent disaster that cost hundreds of millions per year and drove creation of the Alliance for Open Media; dav2d as the planned AV2 decoder. [1] — Jean-Baptiste Kempf "350+ contributors contacted for relicense: To relicense VLC's core from GPL to LGPL, Jean-Baptiste had to personally track down and contact…" 52:24
Claims made here
AV2 delivers approximately 30% better compression than AV1 at equivalent visual quality.
Kyber has achieved 7 milliseconds glass-to-glass video latency from Windows to Windows, with approximately 3.5ms consumed by the NVIDIA hardware encoder and 2ms by the Intel decoder.
Kyber, Jean-Baptiste's startup, has achieved 7 milliseconds glass-to-glass latency for real-time video control, with a target of 4 ms (240 Hz equivalent).
Chapter 24 · 3:54:27
VLC backdoors
Two intelligence agencies requested VLC backdoors and were refused; air-gapped build machines; double-signature verification; detected state-actor fake binary injection; the sandboxing architecture challenge. [1] — Kieran Kunhya "For decades, video codec engineers optimized for PSNR — a purely mathematical metric that made video look blurry. Hobbyist anime fans ignor…" 2:56:04
At least two intelligence agencies approached VideoLAN requesting backdoors in VLC. The answer was an unambiguous no. VLC compiles on air-gapped machines, starting from scratch with the compiler itself, uses double-signature verification, and has caught what appears to be a state actor attempting to inject fake binaries into their servers.
Chapter 26 · 4:11:04
Future of FFmpeg and VLC
VLC and FFmpeg expanding to volumetric video, XR streaming, haptic tracks, spatial audio, brainwave codecs; JB's philosophy on regret; closing tribute to open-source engineers from Linus Torvalds quote.
No indexed bits in this chapter.
Show stoppers
Snapshots ()
Key Quotes ()
This episode
Cast
-
Discussed as the creator of Linux and Git, and as an example of the meritocratic, sometimes harsh code-review culture that FFmpeg and VLC share.
-
The French nonprofit organization behind VLC, dav1d, x264, and numerous other open-source multimedia projects.
-
Track
Discussed as both a major user of FFmpeg (YouTube, Chrome) and the subject of controversy over AI-generated security reports dumped on volunteer FFmpeg maintainers.
-
Track
Discussed as a major FFmpeg user that re-encodes popular videos in AV1, representing approximately 50% of YouTube content.
-
Track
Discussed in the context of Azure running Linux, and the infamous Microsoft Teams bug tracker incident where a manager demanded urgent volunteer support.
-
The consortium of Google, Netflix, Amazon, Apple, Mozilla, and VideoLAN that created the royalty-free AV1 and AV2 video codecs.
-
Jean-Baptiste Kempf's new open-source startup building ultra-low latency video infrastructure for robot and drone teleoperation.
-
Track
Cited as a major user of FFmpeg and AV1, with approximately 30% of its content now delivered in AV1.
-
The French engineering school where VLC originated as a student streaming project in the 1990s, and where x264 was also started.
-
Revealed via WikiLeaks Vault 7 to have distributed a modified VLC installer with a hidden data-exfiltration DLL targeting specific individuals.
-
Kieran Kunhya's company that builds broadcasting equipment for sports, famous for writing hand-crafted assembly achieving 62x speedups over C.
-
The open-source multimedia processing library and tool at the center of the episode, discussed as the invisible backbone of all internet video.
-
The open-source media player developed by VideoLAN, discussed alongside its history, philosophy, and technical architecture.
-
The dominant video codec standard discussed as the foundation of internet video and the benchmark against which all newer codecs are measured.
-
A royalty-free video codec from the Alliance for Open Media delivering 40–60% better compression than H.264, increasingly used by YouTube and Netflix.
-
The open-source H.264 encoder developed under VideoLAN, described as a revolutionary project that redefined internet video quality through psychovisual optimization.
-
VideoLAN's AV1 software decoder, containing 240,000 lines of handwritten assembly and widely regarded as a masterpiece of low-level optimization.
-
Referenced repeatedly as a parallel example of volunteer-built open-source infrastructure that powers most of the world's servers.
-
The memory-safe programming language discussed as potentially useful for new projects but inappropriate for rewriting existing assembly-heavy codebases like FFmpeg.
-
NASA's Mars rover that uses FFmpeg to compress images, making FFmpeg a multi-planetary open-source library.
Stats
This episode
Claims & Sources
Factual claims made this episode, and whether a source was named.
Over 90% of video processing workflows online and offline involve FFmpeg in some form.
VLC has been downloaded at least 6.5 billion times.
Video codecs compress raw video by 100x to 1000x compared to uncompressed formats, compared to about 10x for MP3 audio.
Each new generation of video codec delivers approximately 30% better compression at the same visual quality level.
Approximately 45% of video files are not GPU-decodable and require a software fallback.
Hand-written SIMD assembly for pixel format conversion at Open Broadcast Systems runs 62 times faster than equivalent C code.
The dav1d AV1 decoder contains approximately 240,000 lines of handwritten assembly code — more than double FFmpeg's total assembly across all codecs combined.
Approximately 30% of Netflix video content is now delivered in AV1, and approximately 50% of YouTube content is in AV1.
AV1 delivers 40–60% less bandwidth consumption than H.264 at equivalent visual quality.
FFmpeg is likely running on 100 million to potentially 1 billion CPUs simultaneously at any given moment.
AV2 delivers approximately 30% better compression than AV1 at equivalent visual quality.
AV1 encoding requires approximately two orders of magnitude more CPU cycles than H.264 encoding.
FFmpeg is used by NASA's Mars 2020 rover to compress pictures sent back to Earth.
VLC's latest version still supports Windows XP through Windows 11 and iOS 9 through iOS 26.
Kyber has achieved 7 milliseconds glass-to-glass video latency from Windows to Windows, with approximately 3.5ms consumed by the NVIDIA hardware encoder and 2ms by the Intel decoder.
Jean-Baptiste Kempf contacted more than 350 individual contributors to relicense VLC's core from GPL to LGPL.
Connect
Parsed- FFmpeg official site ffmpeg.org/
- VideoLAN (VLC) videolan.org/
- Jean-Baptiste's Website jbkempf.com/
- FFmpeg FATE testing fate.ffmpeg.org
- Kyber (JB's startup) kyber.io
- <b>Feedback</b> – give feedback to … lexfridman.com/survey
- <b>AMA</b> – submit questions, vide… lexfridman.com/ama
- <b>Hiring</b> – join our team: <a h… lexfridman.com/hiring
- <b>Other</b> – other ways to get in… lexfridman.com/contact
- Lexfridman lexfridman.com/sponsors…
- Kieran’s LinkedIn: <a href=" bit.ly/3OORhmC
- Go to <a href=" lexfridman.com/s/larrid…
- Go to <a href=" lexfridman.com/s/blitzy…
- Go to <a href=" lexfridman.com/s/better…
- Go to <a href=" lexfridman.com/s/fin-ep…
- Go to <a href=" lexfridman.com/s/lmnt-e…
- Go to <a href=" lexfridman.com/s/perple…
- Episode Transcript lexfridman.com/ffmpeg-t…