FFmpeg and GStreamer: Where to Use Which, and When to Run Both

Summary

• FFmpeg: ffmpeg / ffprobe CLI and libav* libraries — format conversion, transcoding, filtering, rapid prototyping
• GStreamer: Element–pad–pipeline architecture — 24/7 live streams, low latency, hardware acceleration, in-app integration
• Relationship: Often alternatives for the same job; frequently complements in one facility (direct hybrid via gst-libav)
• Goal: Not to rank FFmpeg against GStreamer, but to show which one fits which problem, and when using both is the better architecture
• Broadcast: While ST 2110 core paths use vendor SDKs, FFmpeg/GStreamer are common in gateways, OTT bridges, monitoring, and transcoders
• Decision: Files/offline and CLI → FFmpeg; continuous live pipelines and embedded services → GStreamer
• Operations model: FFmpeg favors process-oriented workflows (CLI, worker fleet, supervisor restarts); GStreamer favors in-process pipeline orchestration (state machine, bus, dynamic pads)

Note: This article expands the short FFmpeg vs GStreamer section in Real-time video edge processing with Go. Read it alongside Broadcast codecs, SRT gateway with Go, ST 2110 television campus, NMOS control plane, and HLS stream proxy (Go).

1. Introduction: Frameworks, not “a video app”

In live production, CCTV, VOD, or an IP broadcast campus, teams constantly ask: “Should we do this with FFmpeg or GStreamer?”

Both are multimedia processing frameworks: they read compressed or raw media, pass through codecs, apply filters, and write to another format or network protocol. Neither is a codec — they use H.264, AAC, AV1, and others (mostly via external libraries).

The difference is how they make you think and where they shine:

This article focuses on where to use FFmpeg, where to use GStreamer, and where to use both together, not “which is better.”

1.1 Version scope

Examples and API names target FFmpeg 7.x / 8.x and GStreamer 1.x (typical current releases 1.24+ / 1.26+; some LTS distros still ship FFmpeg 6.x and GStreamer 1.20+). Capabilities depend on your build:

Verify with ffmpeg -version and gst-launch-1.0 --version; use gst-inspect-1.0 to confirm elements exist.

2. Architecture: Filter graph vs pipeline

2.1 FFmpeg — libav and the filter graph

FFmpeg is built around these core libraries:

• libavutil — shared helpers, time, memory, pixel formats
• libavformat — containers (MP4, TS, MKV), protocols (file, RTSP, UDP)
• libavcodec — encode/decode
• libavfilter — scale, deinterlace, overlay, color conversion
• libswscale / libswresample — pixel and audio resampling
• libavdevice (optional) — capture devices such as V4L2, DirectShow

The ffmpeg binary wires these into a filter graph and mux/demux chain. Data flows as frames or packets. For file-based inputs, -re throttles read speed to approximate real-time ingestion (useful when simulating a live feed from disk). It does not turn an input into a native real-time source; on true live sources such as RTSP or UDP, -re is usually unnecessary and can even be harmful.

2.2 GStreamer — elements, pads, caps

In GStreamer, each step is an element: rtspsrc, x264enc, hlssink2, filesink. Elements connect via pads; negotiated types are caps (capabilities): resolution, pixel format, codec, sample rate.

Pipelines use a state machine: NULL → READY → PAUSED → PLAYING. Errors, EOS, and warnings arrive on a bus to the application.

2.3 Common ground

Both can:

• Demux / mux
• Encode / decode (often shared or similar backends)
• Handle network protocols and packaging (RTSP, UDP, SRT, HLS, DASH, RTMP — depending on build)
• Use hardware acceleration (NVENC, VAAPI, Quick Sync — plugin/build dependent)

FFmpeg inside GStreamer: the gst-libav plugin pack exposes FFmpeg decoders/encoders as GStreamer elements (avdec_h264, avenc_aac). That is a direct hybrid, not an either/or choice.

3. FFmpeg in depth

3.1 CLI: fast tool, industry default

Most teams meet FFmpeg first as a command-line tool:

1
2
3
4
5

# RTSP → H.264 MP4 (TCP transport; RTSP drops usually handled via supervisor restart)
ffmpeg -rtsp_transport tcp -i rtsp://camera:554/stream1 \
  -c:v libx264 -preset veryfast -crf 23 \
  -c:a aac -b:a 128k \
  -movflags +faststart output.mp4

1
2
3

# Live UDP multicast ingest → SRT output (example gateway)
ffmpeg -i udp://239.1.1.1:5000?localaddr=192.168.10.5 \
  -c copy -f mpegts "srt://target:9000?mode=caller"

1
2

# Metadata and stream layout
ffprobe -v quiet -print_format json -show_streams input.ts

Strengths:

• End-to-end jobs in one line
• Broad format support
• Ideal for scripts, CI, cron, runbooks
• Huge documentation and community knowledge

Weaknesses (for long-lived services):

• Each ffmpeg invocation is a separate process — restart overhead, memory, pipe I/O
• Fine control needs the C API or exec; error handling stays at process level
• Dynamic topology changes (add/remove branches) are awkward from CLI alone

3.2 Library usage

libavformat + libavcodec integration avoids process overhead; OBS, VLC, many commercial transcoders, and cloud VOD workers use this path. In Go, teams usually choose CGO to libav or exec.Command("ffmpeg", ...) — the edge video processing article covers the latter.

3.3 Typical FFmpeg scenarios

4. GStreamer in depth

4.1 Pipeline language: gst-launch

The CLI counterpart is gst-launch-1.0; syntax chains elements with !:

1
2
3
4
5
6

# RTSP → decode → scale → x264 → MPEG-TS → UDP
gst-launch-1.0 -e rtspsrc location=rtsp://camera:554/stream1 latency=100 ! \
  rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! \
  videoscale ! video/x-raw,width=1280,height=720 ! \
  x264enc tune=zerolatency bitrate=4000 ! h264parse ! \
  mpegtsmux ! udpsink host=239.1.1.2 port=5000 auto-multicast=true

-e (proper EOS shutdown) matters for clean teardown in live systems.

4.2 Application API

Products build pipelines in code (C, Python GObject introspection, Rust gstreamer). Bus messages handle errors, state changes, and EOS; runtime element add/remove (decodebin, uridecodebin) is supported.

Strengths:

• Buffer and clock model built for continuous flow
• Plugin ecosystem: WebRTC, DeepStream, V4L2, ALSA, custom appsrc/appsink
• Hardware encoders as first-class elements (nvh264enc, vaapih264enc)
• Single process, low-latency designs

Weaknesses:

• Setup: distro packages, plugin sets, caps negotiation failures
• Learning: pads, caps, ghost pads, bins
• Fewer “one-liner” recipes than FFmpeg; more API/plugin reference

4.3 Typical GStreamer scenarios

5. Same job, two approaches

Examples below are conceptually equivalent; production bitrates, GOP, B-frames, and transport parameters must match project requirements.

5.1 File → HLS (VOD)

FFmpeg:

1
2
3

ffmpeg -i source.mov -c:v libx264 -preset medium -crf 22 -c:a aac -b:a 192k \
  -hls_time 6 -hls_playlist_type vod -hls_segment_filename 'seg_%03d.ts' \
  -f hls master.m3u8

GStreamer (separate video + audio branches):

1
2
3
4
5

gst-launch-1.0 filesrc location=source.mov ! decodebin name=d \
  d. ! queue ! videoconvert ! x264enc ! h264parse ! mux. \
  d. ! queue ! audioconvert ! audioresample ! voaacenc ! aacparse ! mux. \
  mpegtsmux name=mux ! hlssink2 playlist-location=master.m3u8 \
  location=seg_%05d.ts target-duration=6

Note: decodebin exposes dynamic pads; d. ! routes video/audio to separate branches. hlssink2 needs gst-plugins-bad; splitmuxsink is a common alternative. GStreamer AAC examples often use voaacenc for broad tutorial availability; many deployments prefer fdkaacenc (GPL) or avenc_aac (gst-libav) for quality and support.

Live HLS (event stream): Unlike VOD, you need segment deletion and rolling playlist updates.

1
2
3
4

# FFmpeg — live event (example flags)
ffmpeg -i rtsp://source/stream -c copy -f hls -hls_time 4 \
  -hls_flags delete_segments+append_list+omit_endlist+program_date_time \
  -hls_segment_filename 'live_%03d.ts' index.m3u8

Note: -c copy only works when the source is already H.264 + AAC (or another codec pair your HLS segmenter accepts in MPEG-TS). For HEVC, MJPEG, or incompatible audio, transcode instead — e.g. -c:v libx264 -preset veryfast -c:a aac -b:a 128k.

On GStreamer, use splitmuxsink or hlssink2 with a similar model; for an end-to-end proxy see HLS stream proxy (Go).

DASH (FFmpeg — brief):

1
2

ffmpeg -i source.mov -c:v libx264 -c:a aac \
  -f dash -seg_duration 6 -use_template 1 -use_timeline 1 manifest.mpd

DASH (GStreamer — dashsink, gst-plugins-bad):

dashsink creates segments and the MPD internally — do not place mp4mux in front. Wire video and audio directly to dashsink pads:

1
2
3
4

gst-launch-1.0 filesrc location=source.mov ! decodebin name=d \
  d. ! queue ! videoconvert ! x264enc ! h264parse ! ds.video_0 \
  d. ! queue ! audioconvert ! audioresample ! voaacenc ! aacparse ! ds.audio_0 \
  dashsink name=ds mpd-location=manifest.mpd

Note: dashsink request pad templates are usually video_%u, audio_%u, and subtitle_%u; this is why the example uses video_0 and audio_0. Confirm version/package differences with gst-inspect-1.0 dashsink.

5.2 RTSP camera → raw frames (for analysis)

The RTSP examples below assume H.264 over RTP (rtph264depay). For HEVC: rtph265depay ! h265parse ! avdec_h265 (or a hardware decode element). For MPEG-2, use rtpmp2tdepay and the matching parser.

FFmpeg (stdout pipe — edge article pattern):

1
2

ffmpeg -rtsp_transport tcp -i rtsp://camera/stream \
  -f rawvideo -pix_fmt rgb24 -s 640x480 pipe:1

GStreamer:

1
2
3

gst-launch-1.0 rtspsrc location=rtsp://camera/stream ! rtph264depay ! h264parse ! \
  avdec_h264 ! videoconvert ! video/x-raw,format=RGB,width=640,height=480 ! \
  fdsink fd=1

FFmpeg wins on operational simplicity; GStreamer is more natural when the app keeps everything in one process via appsink.

5.3 Stream copy (re-mux) — save CPU

Change container or protocol without touching codecs:

1
2
3
4
5
6
7
8

# FFmpeg
ffmpeg -i udp://239.1.1.1:5000 -c copy -f mpegts output.ts

# GStreamer (video + audio elementary — parsers after demux)
gst-launch-1.0 udpsrc uri=udp://239.1.1.1:5000 ! tsdemux name=d \
  d. ! queue ! h264parse ! mux. \
  d. ! queue ! aacparse ! mux. \
  mpegtsmux name=mux ! filesink location=output.ts

Note: tsdemux exposes dynamic pads. This example assumes the TS carries both H.264 and AAC elementary streams; drop the audio branch if the feed is video-only. Production also uses parsebin or program-specific pad linking. Actual pad names vary by stream and GStreamer version (e.g. d.video_0, d.audio_0); inspect with gst-launch-1.0 -v or GST_DEBUG=2.

On the FFmpeg side, -c copy moves packets without re-encoding. The GStreamer example demuxes then re-muxes via mpegtsmux. Both avoid transcode, but container metadata and timing (PSI, PCR, PAT/PMT) may still change even without re-encoding — do not assume byte-identical output. Vendor-specific passthrough elements may be required for strict contribution chains.

5.4 Audio pipeline (broadcast sync)

Audio matters as much as video for muxing, latency, and loudness:

Surround (5.1-channel) → stereo downmix (FFmpeg):

1
2
3

ffmpeg -i surround_51.mov -map 0:v -c:v copy \
  -af "pan=stereo|FL=FL+0.707*FC+0.707*BL|FR=FR+0.707*FC+0.707*BR" \
  -c:a aac -b:a 192k out_stereo.mov

GStreamer (6ch → stereo → AAC):

There is no standard audiochannelmix element in GStreamer. Reduce channels with audioconvert + caps:

1
2
3
4
5

gst-launch-1.0 filesrc location=surround_51.mov ! decodebin name=d \
  d. ! queue ! audioconvert ! audio/x-raw,channels=2 ! audioresample \
  ! voaacenc ! aacparse ! mux. \
  d. ! queue ! videoconvert ! x264enc ! h264parse ! mux. \
  mpegtsmux name=mux ! filesink location=out.ts

Note: This performs simple channel folding. For explicit surround matrices (5.1 → stereo weights), FFmpeg -af pan is more controllable; advanced cases need LADSPA or custom mixer chains.

Missing audio on live RTSP is often a wrong pad link or caps negotiation failure — confirm stream layout with ffprobe.

6. Performance, latency, and reliability

6.1 Process vs in-process

As noted in Go edge video processing, reading frames from exec ffmpeg adds pipe I/O and process overhead. That is acceptable for many edge cases; at very high frame rates and tight CPU budgets, GStreamer or direct libav is preferable.

6.2 Hardware acceleration

• FFmpeg: -hwaccel, -c:v h264_nvenc, vaapi, qsv — well documented on the CLI
• GStreamer: nvh264enc, vaapih264enc, platform plugins — added as pipeline elements

Which is faster depends on drivers, versions, and pipeline design — “GStreamer is always faster” is not universally true. Throughput and latency for the same workload can swing 2–10×; measure your own inputs instead of declaring a fixed winner. Starting points: GStreamer benchmarking design notes, FFmpeg release notes, and hardware encoder comparisons.

6.2.1 Low-latency checklist

Low latency is not only about choosing a fast encoder; ingest, queues, GOP, muxing, and player buffering must be tuned together.

6.3 Reliability and reconnect

FFmpeg

GStreamer

Example rtspsrc (more resilient ingest):

1
2

gst-launch-1.0 rtspsrc location=rtsp://camera/stream latency=200 \
  tcp-timeout=5000000 drop-on-latency=true ! rtph264depay ! ...

Production reconnect is usually implemented in application code (bus callbacks) or an external supervisor — there is no single CLI flag equivalent to FFmpeg’s -reconnect family. The two stacks offer different ops models, not a feature gap on one side.

• Encoder stall → watchdog on both sides
• Memory leaks → long soak tests; GStreamer single-process profiling is often clearer

6.4 Scaling (throughput)

High-throughput VOD often uses an FFmpeg worker fleet; many synchronized outputs on one host fit GStreamer’s tee + clock model better.

7. Broadcast and IP production context

7.1 ST 2110 and professional IP

In an ST 2110 television campus, the main media path is RTP multicast + PTP, usually via vendor SDKs (e.g. NVIDIA Rivermax, dedicated SMPTE stacks). FFmpeg and GStreamer do not replace that stack; they excel in:

• Gateways: IP broadcast → OTT (HLS/DASH), SRT contribution
• Monitoring: Recording, thumbnails, waveforms
• Lab / test: Signal generation, loops, format conversion
• Smaller facilities: Software transcoders where full IP products are not deployed

7.2 Transport protocols and OTT packaging

RTMP: Still widely used (OBS publish targets, social platform ingest). FFmpeg writes RTMP via libavformat; legacy librtmp dependency is no longer typical. On GStreamer, use flvmux + rtmpsink (or newer rtmp2sink); TLS needs rtmps URLs and matching plugins.

DASH: Standard in OTT/VOD alongside HLS; FFmpeg -f dash emits an MPD; GStreamer’s dashsink (bad plugin) does similar work inside a pipeline.

CMAF / fMP4: Common in 2026 OTT alongside HLS and DASH; FFmpeg fragmented MP4 (-movflags frag_keyframe+empty_moov), GStreamer cmafsink (build-dependent) follows the same idea.

RIST / NDI: Next to SRT in contribution, teams also use RIST (UDP-friendly, open) and NDI (LAN, vendor ecosystem); both sit outside these frameworks’ core and are usually integrated via vendor SDKs or dedicated plugins.

Example chains:

1
2

Camera (RTSP) → FFmpeg/GStreamer (transcode or copy) → SRT → cloud / playout
OBS (RTMP) → CDN ingest → transcode farm (FFmpeg) → HLS/DASH origin

Media server layer: FFmpeg/GStreamer encode/transcode; MediaMTX and nginx-rtmp terminate protocols and republish (they are not transcoders). Example:

1

GStreamer (encode) --RTSP publish--> MediaMTX --HLS--> CDN / player

For DNxHD, XDCAM, ProRes, see broadcast codecs — those often need professional products and licensing; FFmpeg/GStreamer are strongest with common industry codecs (H.264, HEVC, AV1) and general containers. Here, “supported” does not mean “free of patent/licensing obligations”; H.264/HEVC distribution still needs product- and country-specific review.

7.3 When to use vendor SDKs?

High bitrates, kernel bypass, tight PTP alignment, and SMPTE compliance certification call for broadcast SDKs, not generic frameworks. FFmpeg/GStreamer sit at the edge and derivative layers of that world.

7.4 AV1 (practical status in 2026)

AV1 is no longer a “future codec” only — OTT archives and VOD use it in production with both FFmpeg and GStreamer; large-scale or experimental live ingest exists but remains niche next to H.264/HEVC, especially for low-latency contribution. Background: What is AV1?, app notes in AV1 converter, cloud context in AV1 and AWS.

FFmpeg example (SVT-AV1, VOD):

1

ffmpeg -i source.mov -c:v libsvtav1 -preset 6 -crf 30 -c:a copy out.mkv

GStreamer example (SVT-AV1):

1
2
3
4

gst-launch-1.0 filesrc location=source.mov ! decodebin name=d \
  d. ! queue ! videoconvert ! svtav1enc ! av1parse ! mux. \
  d. ! queue ! audioconvert ! voaacenc ! aacparse ! mux. \
  matroskamux name=mux ! filesink location=out.mkv

AV1 is not as universal as H.264/HEVC for broadcast contribution yet, but it is a first-class option for OTT origins, archives, and bandwidth reduction — the same libsvtav1 ideas from AV1 converter apply to CLI and pipeline setups.

7.5 Subtitles and closed captions (brief)

Broadcast and OTT often need CEA-608/708 (line-21 style) and WebVTT (HLS/DASH sidecar):

• FFmpeg: mux SubRip .srt / mov_text, burn-in via subtitles filter; separate subtitle track with -map 0:s
• GStreamer: subtitleoverlay, closedcaption (region/build dependent), WebVTT via HLS side manifest or separate files

Full workflows usually live in playout/MAM; FFmpeg/GStreamer preserve or embed caption tracks at transcode/package time. CEA-608/708 does not always arrive as a normal subtitle stream; depending on MPEG-TS, H.264 SEI, MXF, or side-data representation, mapping, bitstream filters, and container support must be tested separately.

8. Go and application integration

8.1 FFmpeg + Go

Common pattern:

1
2
3
4
5
6
7
8

cmd := exec.CommandContext(ctx, "ffmpeg",
    "-rtsp_transport", "tcp",
    "-i", rtspURL,
    "-f", "rawvideo", "-pix_fmt", "rgb24",
    "pipe:1",
)
stdout, _ := cmd.StdoutPipe()
// frame read loop

Pros: fast development, independent FFmpeg upgrades. Cons: process management, stderr logging, graceful shutdown.

8.2 GStreamer + Go (and Python)

• CGO / go-gst: Pipeline in-process; less process overhead
• Python (gi.repository.Gst): Common for prototypes and ML glue; same bus/state model
• Separate GStreamer service + gRPC/SHM: Language split, GPU nodes

Many edge projects validate with FFmpeg CLI first, then move to GStreamer or libav when requirements crystallize — matching the choice in edge processing.

9. Decision tree

Quick guide:

9.1 Quick selection recipes

10. Licensing, platforms, and operations

10.1 Licensing (critical for commercial products)

“We use FFmpeg” is not enough for broadcast products: auditors ask which build, which codecs, and static vs dynamic linking. GStreamer’s gst-plugins-ugly may ship patented/GPL components as separate packages.

Audio codec note: Examples use aac / voaacenc for portability; broadcast loudness and quality often need libfdk-aac (non-free build/licensing impact on the FFmpeg side) or GStreamer fdkaacenc / avenc_aac — plan licensing together with §10.1.

10.2 Platforms: Linux, Windows, macOS

Cross-platform teams often standardize on FFmpeg for scripts and ops everywhere; GStreamer for always-on live services is usually Linux-targeted.

10.3 Containers and debug

• Docker: Images such as jrottenberg/ffmpeg are common for ops transcode; GStreamer images typically need custom Dockerfiles with gst-*-plugins-* dev packages.
• GStreamer debug: GST_DEBUG=3, gst-inspect-1.0 rtspsrc, gst-launch-1.0 -v — first tools for caps/pad failures.
• Pipeline visualization: GST_DEBUG_DUMP_DOT_DIR=./dots — runtime .dot graphs (view with Graphviz); gst-shark (if installed) for latency/throughput profiling.
• FFmpeg diagnosis: ffprobe -show_streams, -loglevel debug.

10.4 Installation verification commands

Before running the article examples, verify that your build actually includes the relevant protocol, encoder, and GStreamer element:

1
2
3
4

ffmpeg -hide_banner -version
ffmpeg -hide_banner -protocols | grep -E 'srt|rtmp|rtsp|http'
ffmpeg -hide_banner -encoders | grep -E 'libx264|libsvtav1|nvenc|qsv|vaapi'
ffprobe -hide_banner -show_streams input.ts

1
2
3
4
5

gst-launch-1.0 --version
gst-inspect-1.0 rtspsrc
gst-inspect-1.0 hlssink2
gst-inspect-1.0 dashsink
gst-inspect-1.0 svtav1enc

These checks matter especially in container images, CI runners, and embedded devices; a command can work on your laptop while the production image is missing a plugin.

10.5 Quick troubleshooting

11. Other tools (brief)

Beyond FFmpeg and GStreamer:

• libVLC — playback and simple streams; embedded players
• MediaMTX / nginx-rtmp — protocol servers; termination layer in the chain above
• OBS — uses FFmpeg/libav components internally, but is architecturally its own application stack (libobs, plugins); not a thin wrapper around the ffmpeg CLI
• Go-native (gortsplib, pure Go codecs) — fewer dependencies; limited format/codec coverage

Edge processing discusses Go-native options. Format breadth and ops tooling still favor FFmpeg in OTT, CDN, and VOD/transcode farms. In embedded devices, video analytics, and GPU-centric live pipelines, GStreamer (and ecosystems such as NVIDIA DeepStream, which is GStreamer-based) are equally entrenched — dominance is domain-specific, not universal.

12. Conclusion

FFmpeg and GStreamer offer different architectural choices, not just two brands:

• FFmpeg = universal multimedia tool and library; fast, documented, everywhere
• GStreamer = modular pipeline engine; live, embedded, dynamic graphs

Most organizations run both: FFmpeg for archives and ops scripts, GStreamer (or gst-libav hybrids) for live products. In broadcast IT, the sharper split is ST 2110 core vs software gateway — these frameworks are critical in the bridge and operations layer.

The right question is not “FFmpeg or GStreamer?” but “File or stream? Is CLI enough or do we need in-process? Campus core or edge?”

References

• FFmpeg Documentation
• GStreamer Documentation
• gst-libav
• Related posts: Go edge video, SRT + Go, HLS proxy (Go), Broadcast codecs, ST 2110 campus, NMOS, AV1, AV1 converter, AV1 and AWS — content folder AV1andAWS, published URL /av1andaws/