gstadaptivedemux
What is an adaptive demuxer? Adaptive demuxers are special demuxers in the sense that they don't actually demux data received from upstream but download the data themselves.
Adaptive formats (HLS, DASH, MSS) are composed of a manifest file and a set of fragments. The manifest describes the available media and the sequence of fragments to use. Each fragment contains a small part of the media (typically only a few seconds). It is possible for the manifest to have the same media available in different configurations (bitrates for example) so that the client can select the one that best suits its scenario (network fluctuation, hardware requirements...). It is possible to switch from one representation of the media to another during playback. That's why it is called 'adaptive', because it can be adapted to the client's needs.
Architectural overview: The manifest is received by the demuxer in its sink pad and, upon receiving EOS, it parses the manifest and exposes the streams available in it. For each stream a source element will be created and will download the list of fragments one by one. Once a fragment is finished downloading, the next URI is set to the source element and it starts fetching it and pushing through the stream's pad. This implies that each stream is independent from each other as it runs on a separate thread.
After downloading each fragment, the download rate of it is calculated and the demuxer has a chance to switch to a different bitrate if needed. The switch can be done by simply pushing a new caps before the next fragment when codecs are the same, or by exposing a new pad group if it needs a codec change.
Extra features:
- Not linked streams: Streams that are not-linked have their download threads interrupted to save network bandwidth. When they are relinked a reconfigure event is received and the stream is restarted.
Subclasses: While GstAdaptiveDemux is responsible for the workflow, it knows nothing about the intrinsics of the subclass formats, so the subclasses are responsible for maintaining the manifest data structures and stream information.
Functions
gst_adaptive_demux_find_stream_for_pad
GstAdaptiveDemuxStream * gst_adaptive_demux_find_stream_for_pad (GstAdaptiveDemux * demux, GstPad * pad)
Parameters:
demux
–
pad
–
gst_adaptive_demux_get_client_now_utc
GDateTime * gst_adaptive_demux_get_client_now_utc (GstAdaptiveDemux * demux)
Used to find the client's estimate of UTC, using the system realtime clock.
Parameters:
demux
–
GstAdaptiveDemux Returns: the client's estimate of UTC
gst_adaptive_demux_get_monotonic_time
GstClockTime gst_adaptive_demux_get_monotonic_time (GstAdaptiveDemux * demux)
Parameters:
demux
–
a monotonically increasing time, using the system realtime clock
gst_adaptive_demux_get_qos_earliest_time
GstClockTime gst_adaptive_demux_get_qos_earliest_time (GstAdaptiveDemux * demux)
Parameters:
demux
–
The QOS earliest time
Since : 1.20
gst_adaptive_demux_is_running
gboolean gst_adaptive_demux_is_running (GstAdaptiveDemux * demux)
Returns FALSE if shutdown has started (transitioning down from PAUSED), otherwise TRUE.
Parameters:
demux
–
GstAdaptiveDemux Returns: whether the demuxer is processing data
gst_adaptive_demux_set_stream_struct_size
void gst_adaptive_demux_set_stream_struct_size (GstAdaptiveDemux * demux, gsize struct_size)
Parameters:
demux
–
struct_size
–
gst_adaptive_demux_stream_advance_fragment
GstFlowReturn gst_adaptive_demux_stream_advance_fragment (GstAdaptiveDemux * demux, GstAdaptiveDemuxStream * stream, GstClockTime duration)
Parameters:
demux
–
stream
–
duration
–
gst_adaptive_demux_stream_fragment_clear
void gst_adaptive_demux_stream_fragment_clear (GstAdaptiveDemuxStreamFragment * f)
Parameters:
f
–
gst_adaptive_demux_stream_new
GstAdaptiveDemuxStream * gst_adaptive_demux_stream_new (GstAdaptiveDemux * demux, GstPad * pad)
Parameters:
demux
–
pad
–
gst_adaptive_demux_stream_push_buffer
GstFlowReturn gst_adaptive_demux_stream_push_buffer (GstAdaptiveDemuxStream * stream, GstBuffer * buffer)
Parameters:
stream
–
buffer
–
gst_adaptive_demux_stream_queue_event
void gst_adaptive_demux_stream_queue_event (GstAdaptiveDemuxStream * stream, GstEvent * event)
Parameters:
stream
–
event
–
gst_adaptive_demux_stream_set_caps
void gst_adaptive_demux_stream_set_caps (GstAdaptiveDemuxStream * stream, GstCaps * caps)
Parameters:
stream
–
caps
–
gst_adaptive_demux_stream_set_tags
void gst_adaptive_demux_stream_set_tags (GstAdaptiveDemuxStream * stream, GstTagList * tags)
Parameters:
stream
–
tags
–
Structures
GstAdaptiveDemuxClass
struct _GstAdaptiveDemuxClass { /** * process_manifest: Parse the manifest * @demux: #GstAdaptiveDemux * @manifest: the manifest to be parsed * * Parse the manifest and add the created streams using * gst_adaptive_demux_stream_new() * * Returns: %TRUE if successful */ gboolean (*process_manifest) (GstAdaptiveDemux * demux, GstBuffer * manifest); /** * get_manifest_update_interval: * @demux: #GstAdaptiveDemux * * Used during live streaming, the subclass should return the interval * between successive manifest updates * * Returns: the update interval in microseconds */ gint64 (*get_manifest_update_interval) (GstAdaptiveDemux * demux); /** * update_manifest: * @demux: #GstAdaptiveDemux * * During live streaming, this will be called for the subclass to update its * manifest with the new version. By default it fetches the manifest URI * and passes it to GstAdaptiveDemux::update_manifest_data(). * * Returns: #GST_FLOW_OK is all succeeded, #GST_FLOW_EOS if the stream ended * or #GST_FLOW_ERROR if an error happened */ GstFlowReturn (*update_manifest) (GstAdaptiveDemux * demux); /** * update_manifest_data: * @demux: #GstAdaptiveDemux * @buf: Downloaded manifest data * * During live streaming, this will be called for the subclass to update its * manifest with the new version * * Returns: #GST_FLOW_OK is all succeeded, #GST_FLOW_EOS if the stream ended * or #GST_FLOW_ERROR if an error happened */ GstFlowReturn (*update_manifest_data) (GstAdaptiveDemux * demux, GstBuffer * buf); gboolean (*is_live) (GstAdaptiveDemux * demux); GstClockTime (*get_duration) (GstAdaptiveDemux * demux); /** * reset: * @demux: #GstAdaptiveDemux * * Reset the internal state of the subclass, getting ready to restart with * a new stream afterwards */ void (*reset) (GstAdaptiveDemux * demux); /** * seek: * @demux: #GstAdaptiveDemux * @seek: a seek #GstEvent * * The demuxer should seek on all its streams to the specified position * in the seek event * * Returns: %TRUE if successful */ gboolean (*seek) (GstAdaptiveDemux * demux, GstEvent * seek); /** * has_next_period: * @demux: #GstAdaptiveDemux * * Checks if there is a next period following the current one. * DASH can have multiple medias chained in its manifest, when one finishes * this function is called to verify if there is a new period to be played * in sequence. * * Returns: %TRUE if there is another period */ gboolean (*has_next_period) (GstAdaptiveDemux * demux); /** * advance_period: * @demux: #GstAdaptiveDemux * * Advances the manifest to the next period. New streams should be created * using gst_adaptive_demux_stream_new(). */ void (*advance_period) (GstAdaptiveDemux * demux); void (*stream_free) (GstAdaptiveDemuxStream * stream); GstFlowReturn (*stream_seek) (GstAdaptiveDemuxStream * stream, gboolean forward, GstSeekFlags flags, GstClockTime target_ts, GstClockTime * final_ts); gboolean (*stream_has_next_fragment) (GstAdaptiveDemuxStream * stream); GstFlowReturn (*stream_advance_fragment) (GstAdaptiveDemuxStream * stream); /** * need_another_chunk: * @stream: #GstAdaptiveDemuxStream * * If chunked downloading is used (chunk_size != 0) this is called once a * chunk is finished to decide whether more has to be downloaded or not. * May update chunk_size to a different value */ gboolean (*need_another_chunk) (GstAdaptiveDemuxStream * stream); /** * stream_update_fragment_info: * @stream: #GstAdaptiveDemuxStream * * Requests the stream to set the information about the current fragment to its * current fragment struct * * Returns: #GST_FLOW_OK in success, #GST_FLOW_ERROR on error and #GST_FLOW_EOS * if there is no fragment. */ GstFlowReturn (*stream_update_fragment_info) (GstAdaptiveDemuxStream * stream); /** * stream_select_bitrate: * @stream: #GstAdaptiveDemuxStream * @bitrate: the bitrate to select (in bytes per second) * * The stream should try to select the bitrate that is the greater, but not * greater than the requested bitrate. If it needs a codec change it should * create the new stream using gst_adaptive_demux_stream_new(). If it only * needs a caps change it should set the new caps using * gst_adaptive_demux_stream_set_caps(). * * Returns: %TRUE if the stream changed bitrate, %FALSE otherwise */ gboolean (*stream_select_bitrate) (GstAdaptiveDemuxStream * stream, guint64 bitrate); /** * stream_get_fragment_waiting_time: * @stream: #GstAdaptiveDemuxStream * * For live streams, requests how much time should be waited before starting * to download the fragment. This is useful to avoid downloading a fragment that * isn't available yet. * * Returns: The waiting time in microseconds */ gint64 (*stream_get_fragment_waiting_time) (GstAdaptiveDemuxStream * stream); /** * start_fragment: * @demux: #GstAdaptiveDemux * @stream: #GstAdaptiveDemuxStream * * Notifies the subclass that the given stream is starting the download * of a new fragment. Can be used to reset/init internal state that is * needed before each fragment, like decryption engines. * * Returns: %TRUE if successful. */ gboolean (*start_fragment) (GstAdaptiveDemux * demux, GstAdaptiveDemuxStream * stream); /** * finish_fragment: * @demux: #GstAdaptiveDemux * @stream: #GstAdaptiveDemuxStream * * Notifies the subclass that a fragment download was finished. * It can be used to cleanup internal state after a fragment and * also push any pending data before moving to the next fragment. */ GstFlowReturn (*finish_fragment) (GstAdaptiveDemux * demux, GstAdaptiveDemuxStream * stream); /** * data_received: * @demux: #GstAdaptiveDemux * @stream: #GstAdaptiveDemuxStream * @buffer: #GstBuffer * * Notifies the subclass that a fragment chunk was downloaded. The subclass * can look at the data and modify/push data as desired. * * Returns: #GST_FLOW_OK if successful, #GST_FLOW_ERROR in case of error. */ GstFlowReturn (*data_received) (GstAdaptiveDemux * demux, GstAdaptiveDemuxStream * stream, GstBuffer * buffer); /** * get_live_seek_range: * @demux: #GstAdaptiveDemux * @start: pointer to put the start position allowed to seek to * @stop: pointer to put the stop position allowed to seek to * * Gets the allowed seek start and stop positions for the current live stream * * Return: %TRUE if successful */ gboolean (*get_live_seek_range) (GstAdaptiveDemux * demux, gint64 * start, gint64 * stop); /** * get_presentation_offset: * @demux: #GstAdaptiveDemux * @stream: #GstAdaptiveDemuxStream * * Gets the delay to apply to @stream. * * Return: a #GstClockTime representing the (positive) time offset to apply to * @stream. */ GstClockTime (*get_presentation_offset) (GstAdaptiveDemux *demux, GstAdaptiveDemuxStream *stream); /** * get_period_start_time: * @demux: #GstAdaptiveDemux * * Gets the start time of the current period. Timestamps are resetting to 0 * after each period but we have to maintain a continuous stream and running * time so need to know the start time of the current period. * * Return: a #GstClockTime representing the start time of the currently * selected period. */ GstClockTime (*get_period_start_time) (GstAdaptiveDemux *demux); /** * requires_periodical_playlist_update: * @demux: #GstAdaptiveDemux * * Some adaptive streaming protocols allow the client to download * the playlist once and build up the fragment list based on the * current fragment metadata. For those protocols the demuxer * doesn't need to periodically refresh the playlist. This vfunc * is relevant only for live playback scenarios. * * Return: %TRUE if the playlist needs to be refreshed periodically by the demuxer. */ gboolean (*requires_periodical_playlist_update) (GstAdaptiveDemux * demux); };
Fields
process_manifest
()
–
get_manifest_update_interval
()
–
update_manifest
()
–
update_manifest_data
()
–
is_live
()
–
get_duration
()
–
reset
()
–
seek
()
–
has_next_period
()
–
advance_period
()
–
stream_free
()
–
stream_seek
()
–
stream_has_next_fragment
()
–
stream_advance_fragment
()
–
need_another_chunk
()
–
stream_update_fragment_info
()
–
stream_select_bitrate
()
–
stream_get_fragment_waiting_time
()
–
start_fragment
()
–
finish_fragment
()
–
data_received
()
–
get_live_seek_range
()
–
get_presentation_offset
()
–
get_period_start_time
()
–
requires_periodical_playlist_update
()
–
GstAdaptiveDemuxStream
struct _GstAdaptiveDemuxStream { GstPad *pad; GstPad *internal_pad; GstAdaptiveDemux *demux; GstSegment segment; GstCaps *pending_caps; GstEvent *pending_segment; GstTagList *pending_tags; gboolean need_header; GList *pending_events; GstFlowReturn last_ret; GError *last_error; GstTask *download_task; GRecMutex download_lock; gboolean restart_download; gboolean discont; gboolean downloading_first_buffer; gboolean downloading_header; gboolean downloading_index; gboolean bitrate_changed; /* download tooling */ GstElement *src; guint last_status_code; GstPad *src_srcpad; GstElement *uri_handler; GstElement *queue; GMutex fragment_download_lock; GCond fragment_download_cond; gboolean download_finished; /* protected by fragment_download_lock */ gboolean cancelled; /* protected by fragment_download_lock */ gboolean replaced; /* replaced in a bitrate switch (used with cancelled) */ gboolean src_at_ready; /* protected by fragment_download_lock */ gboolean starting_fragment; gboolean first_fragment_buffer; gint64 download_start_time; gint64 download_total_bytes; guint64 current_download_rate; /* amount of data downloaded in current fragment (pre-queue2) */ guint64 fragment_bytes_downloaded; /* bitrate of the previous fragment (pre-queue2) */ guint64 last_bitrate; /* latency (request to first byte) and full download time (request to EOS) * of previous fragment (pre-queue2) */ GstClockTime last_latency; GstClockTime last_download_time; /* Average for the last fragments */ guint64 moving_bitrate; guint moving_index; guint64 *fragment_bitrates; /* QoS data : UNUSED !!! */ GstClockTime qos_earliest_time; GstAdaptiveDemuxStreamFragment fragment; guint download_error_count; /* TODO check if used */ gboolean eos; gboolean do_block; /* TRUE if stream should block on preroll */ };
Fields
pad
(GstPad *)
–
internal_pad
(GstPad *)
–
demux
(GstAdaptiveDemux *)
–
segment
(GstSegment)
–
pending_caps
(GstCaps *)
–
pending_segment
(GstEvent *)
–
pending_tags
(GstTagList *)
–
need_header
(gboolean)
–
pending_events
(GList *)
–
last_ret
(GstFlowReturn)
–
last_error
(GError *)
–
download_task
(GstTask *)
–
download_lock
(GRecMutex)
–
restart_download
(gboolean)
–
discont
(gboolean)
–
downloading_first_buffer
(gboolean)
–
downloading_header
(gboolean)
–
downloading_index
(gboolean)
–
bitrate_changed
(gboolean)
–
src
(GstElement *)
–
last_status_code
(guint)
–
src_srcpad
(GstPad *)
–
uri_handler
(GstElement *)
–
queue
(GstElement *)
–
fragment_download_lock
(GMutex)
–
fragment_download_cond
(GCond)
–
download_finished
(gboolean)
–
cancelled
(gboolean)
–
replaced
(gboolean)
–
src_at_ready
(gboolean)
–
starting_fragment
(gboolean)
–
first_fragment_buffer
(gboolean)
–
download_start_time
(gint64)
–
download_total_bytes
(gint64)
–
current_download_rate
(guint64)
–
fragment_bytes_downloaded
(guint64)
–
last_bitrate
(guint64)
–
last_latency
(GstClockTime)
–
last_download_time
(GstClockTime)
–
moving_bitrate
(guint64)
–
moving_index
(guint)
–
fragment_bitrates
(guint64 *)
–
qos_earliest_time
(GstClockTime)
–
fragment
(GstAdaptiveDemuxStreamFragment)
–
download_error_count
(guint)
–
eos
(gboolean)
–
do_block
(gboolean)
–
GstAdaptiveDemuxStreamFragment
struct _GstAdaptiveDemuxStreamFragment { GstClockTime timestamp; GstClockTime duration; gchar *uri; gint64 range_start; gint64 range_end; /* when chunked downloading is used, may be be updated need_another_chunk() */ guint chunk_size; /* when headers are needed */ gchar *header_uri; gint64 header_range_start; gint64 header_range_end; /* when index is needed */ gchar *index_uri; gint64 index_range_start; gint64 index_range_end; /* Nominal bitrate as provided by * sub-class or calculated by base-class */ guint bitrate; gboolean finished; };
Fields
timestamp
(GstClockTime)
–
duration
(GstClockTime)
–
uri
(gchar *)
–
range_start
(gint64)
–
range_end
(gint64)
–
chunk_size
(guint)
–
header_uri
(gchar *)
–
header_range_start
(gint64)
–
header_range_end
(gint64)
–
index_uri
(gchar *)
–
index_range_start
(gint64)
–
index_range_end
(gint64)
–
bitrate
(guint)
–
finished
(gboolean)
–
The results of the search are