Inside the engine
RTC, frame/sample access and GPU.
Connected through CoreGraph.
Connect real-time media, custom audio/video processing and CoreGL graphics in one native graph. CoreGraph handles routing, timing and lifecycle across live and offline workflows.
Need a specific capability? Browse the searchable feature index ↗
Protocols and delivery topologies
Connect RTC to your media pipeline.
- RTC / WebRTC
- Send, receive and relay media with browser publication/viewing, signaling and configurable ICE/STUN/TURN connectivity.
- SFU forwarding
- Forward compatible encoded tracks from multiple publishers to selected viewers. Ingest conversion is separate.
- MCU composition
- Decode and compose selected inputs, mix audio, then publish or record a program, including output back through SFU.
- File and network I/O
- RTMP(S), RTSP, SRT, HTTP(S), RTP and other file/network paths available in the selected CoreMF build.
- HLS / DASH / LL-HLS
- HLS/DASH bitrate ladders and dedicated LL-HLS parts, playlists and HTTP serving.
Protocol availability follows the selected build. Server workflows target Linux; dynamic server MCU is development/reference and WHIP is preview. Media-server solution ↗
Video frame and audio sample access through multi-language plugins
Extend processing inside the media graph.
Access individual video frames and audio sample buffers through runtime-loaded native, Python or JavaScript plugins. Add custom filters or AI while CoreGraph manages routing, timing and lifecycle. Packet access, buffer layout and ownership follow the matching plugin contract.
Use a selected AI processor/model to return modified media or timestamped results that drive GPU overlays, composition and application decisions.
Native plugin authoring uses the agreed development headers/ABI. The binary application SDK loads/configures plugins but does not expose arbitrary raw-frame pointers. No default AI model pack is included. Compare interfaces ↗
Integration choices
One engine. Your language and platform.
- Program or configure
- Build graphs through API calls or Graph JSON. Reuse templates, compose subgraphs and apply supported runtime changes.
- Application languages
- C/C++, Python, Node.js, Java and Go use the same native engine through the public C ABI. Plugin authoring uses its matching development interface.
- Platform builds
- Linux/Ubuntu, macOS and Windows. Check the delivered platform scope; current installed-package evidence identifies Ubuntu 24.04, and server/container delivery is Linux-specific.
- Component reuse
- Native OO classes retain their ownership and dependencies. Agreed source/component delivery supports focused integrations without adopting a reference application.
CoreGL
GPU processing inside the media path.
GPU composition, filters and production graphics.
Compose inputs with CoreGL shaders, filters, titles, LUTs, keying, transitions and PiP/grid layouts. Keyframes and timestamped AI or application metadata can drive layers and effects.
Select the GPU runtime and CPU/GPU boundaries for your pipeline. GPU composition, hardware encoding and inference are separate capabilities; measure transfers and driver behavior on the target.
Codec performance & encoder choice
Native codecs. Flexible encoder selection.
Native media processing supports software and available hardware decode/encode paths. Automatic selection tries eligible encoders for the requested codec, respecting availability, GPL policy and configured fallback. Choose a path for your latency and throughput targets; performance depends on the codec, device and workload.
Optional BYO encoding explicitly invokes a customer executable at
runtime: CPU frames go out and compressed packets return for native
muxing or streaming. Adapter/platform scope is qualified separately.
Windows also has an explicit native H.264 encoding path
(-mft).
Reproducible composition example
Real sports. Four sources. One CoreGL grid.
Volleyball, bouldering, lead climbing and body-surfing footage enter four decoder paths. CoreGL composes the pictures into a 1280×720 grid, then CoreMF encodes the fourteen-second H.264 MP4 through the application API.
What this example demonstrates
Four decoder paths feed GPU composition and native encoding. Configuration, source credits and test conditions are linked below.
Offline development output; not RTC/MCU, synchronization or performance qualification.
Footage: Shev123, Voltmetro and JaredMcKenzie · CC0. No endorsement is implied.
Footage credits, graph configuration and test conditions ↗CoreAudio
Supporting audio processing.
CoreAudio provides configurable mixing, DSP and monitoring chains alongside video processing. It is still maturing; select and validate the effects your workflow needs.
Protection levels
Media payload encryption and protected packaging.
- StreamCrypto: packets
- Graph nodes encrypt/decrypt compressed packets with AES-256-GCM and key/nonce metadata.
- StreamCrypto: frames
- Native helpers protect serialized decoded frames; packet graph nodes do not automatically encrypt raw frames.
- StreamCrypto: segments
- Helpers protect configured segment payloads and LL-HLS integration paths, separately from standard DRM packaging.
- DRM: samples and delivery
- CENC/CBCS and HLS protected profiles, packaging and key signaling for compatible H.264, HEVC and AAC media.
Media protection is separate from TLS/RTC transport security. Your key/license authority supplies authorization and provider/CDM integration; player/provider qualification remains separate. Qualification scope ↗
More than a transcode command.
The API-based runner exposes the same native engine for configured pipelines, templates, plugins and runtime inspection.
Download the runner capability guide ↗Build with CoreMF
Check the supported configuration.
Check qualification and known limits for the features your application requires.