rtpjitterbuffer

This element reorders and removes duplicate RTP packets as they are received from a network source.

The element needs the clock-rate of the RTP payload in order to estimate the delay. This information is obtained either from the caps on the sink pad or, when no caps are present, from the request-pt-map signal. To clear the previous pt-map use the clear-pt-map signal.

The rtpjitterbuffer will wait for missing packets up to a configurable time limit using the latency property. Packets arriving too late are considered to be lost packets. If the do-lost property is set, lost packets will result in a custom serialized downstream event of name GstRTPPacketLost. The lost packet events are usually used by a depayloader or other element to create concealment data or some other logic to gracefully handle the missing packets.

The jitterbuffer will use the DTS (or PTS if no DTS is set) of the incoming buffer and the rtptime inside the RTP packet to create a PTS on the outgoing buffer.

The jitterbuffer can also be configured to send early retransmission events upstream by setting the do-retransmission property. In this mode, the jitterbuffer tries to estimate when a packet should arrive and sends a custom upstream event named GstRTPRetransmissionRequest when the packet is considered late. The initial expected packet arrival time is calculated as follows:

If seqnum N arrived at time T, seqnum N+1 is expected to arrive at T + packet-spacing + rtx-delay. The packet spacing is calculated from the DTS (or PTS is no DTS) of two consecutive RTP packets with different rtptime.
If seqnum N0 arrived at time T0 and seqnum Nm arrived at time Tm, seqnum Ni is expected at time Ti = T0 + i*(Tm - T0)/(Nm - N0). Any previously scheduled timeout is overwritten.
If seqnum N arrived, all seqnum older than N - rtx-delay-reorder are considered late immediately. This is to request fast feedback for abnormally reorder packets before any of the previous timeouts is triggered.

A late packet triggers the GstRTPRetransmissionRequest custom upstream event. After the initial timeout expires and the retransmission event is sent, the timeout is scheduled for T + rtx-retry-timeout. If the missing packet did not arrive after rtx-retry-timeout, a new GstRTPRetransmissionRequest is sent upstream and the timeout is rescheduled again for T + rtx-retry-timeout. This repeats until rtx-retry-period elapsed, at which point no further retransmission requests are sent and the regular logic is performed to schedule a lost packet as discussed above.

This element acts as a live element and so adds latency to the pipeline.

This element will automatically be used inside rtpbin.

Example pipelines

 gst-launch-1.0 rtspsrc location=rtsp://192.168.1.133:8554/mpeg1or2AudioVideoTest ! rtpjitterbuffer ! rtpmpvdepay ! mpeg2dec ! xvimagesink

Connect to a streaming server and decode the MPEG video. The jitterbuffer is inserted into the pipeline to smooth out network jitter and to reorder the out-of-order RTP packets.

Hierarchy

GObject
    ╰──GInitiallyUnowned
        ╰──GstObject
            ╰──GstElement
                ╰──rtpjitterbuffer

Factory details

Authors: – Philippe Kalaf , Wim Taymans

Classification: – Filter/Network/RTP

Rank – none

Plugin – rtpmanager

Package – GStreamer Good Plug-ins

Pad Templates

`sink`

application/x-rtp:

Presence – always

Direction – sink

Object type – GstPad

`sink_rtcp`

application/x-rtcp:

Presence – request

Direction – sink

Object type – GstPad

`src`

application/x-rtp:

Presence – always

Direction – src

Object type – GstPad

Signals

handle-sync

handle_sync_callback (GstElement * buffer,
                      GstStructure * struct,
                      gpointer udata)

def handle_sync_callback (buffer, struct, udata):
    #python callback for the 'handle-sync' signal

function handle_sync_callback(buffer: GstElement * buffer, struct: GstStructure * struct, udata: gpointer udata): {
    // javascript callback for the 'handle-sync' signal
}

Be notified of new sync values.

Parameters:

buffer –

the object which received the signal

struct –

a GstStructure containing sync values.

udata –

No description available

Flags: Run Last

on-npt-stop

on_npt_stop_callback (GstElement * buffer,
                      gpointer udata)

def on_npt_stop_callback (buffer, udata):
    #python callback for the 'on-npt-stop' signal

function on_npt_stop_callback(buffer: GstElement * buffer, udata: gpointer udata): {
    // javascript callback for the 'on-npt-stop' signal
}

Signal that the jitterbuffer has pushed the RTP packet that corresponds to the npt-stop position.

Parameters:

buffer –

the object which received the signal

udata –

No description available

Flags: Run Last

request-pt-map

GstCaps *
request_pt_map_callback (GstElement * buffer,
                         guint pt,
                         gpointer udata)

def request_pt_map_callback (buffer, pt, udata):
    #python callback for the 'request-pt-map' signal

function request_pt_map_callback(buffer: GstElement * buffer, pt: guint pt, udata: gpointer udata): {
    // javascript callback for the 'request-pt-map' signal
}

Request the payload type as GstCaps for pt.

Parameters:

buffer –

the object which received the signal

pt –

the pt

udata –

No description available

Returns (GstCaps *) –

No description available

Flags: Run Last

Action Signals

clear-pt-map

g_signal_emit_by_name (buffer, "clear-pt-map");

ret = buffer.emit ("clear-pt-map")

let ret = buffer.emit ("clear-pt-map");

Invalidate the clock-rate as obtained with the request-pt-map signal.

Parameters:

buffer (GstElement *) –

the object which received the signal

Flags: Run Last / Action

set-active

g_signal_emit_by_name (buffer, "set-active", arg0, arg1, &ret);

ret = buffer.emit ("set-active", arg0, arg1)

let ret = buffer.emit ("set-active", arg0, arg1);

Start pushing out packets with the given base time. This signal is only useful in buffering mode.

Parameters:

buffer (GstElement *) –

the object which received the signal

arg0 (gboolean) –

No description available

arg1 (guint64) –

No description available

Returns (guint64) –

the time of the last pushed packet.

Flags: Run Last / Action

Properties

add-reference-timestamp-meta

“add-reference-timestamp-meta” gboolean

When syncing to a RFC7273 clock or after clock synchronization via RTCP or inband NTP-64 header extensions has happened, add GstReferenceTimestampMeta to buffers with the original reconstructed reference clock timestamp.

Flags : Read / Write

Default value : false

Since : 1.22

do-lost

“do-lost” gboolean

Send out a GstRTPPacketLost event downstream when a packet is considered lost.

Flags : Read / Write

Default value : false

do-retransmission

“do-retransmission” gboolean

Send out a GstRTPRetransmission event upstream when a packet is considered late and should be retransmitted.

Flags : Read / Write

Default value : false

Since : 1.2

drop-messages-interval

“drop-messages-interval” guint

Minimal time in milliseconds between posting dropped packet messages, if enabled by setting property by setting post-drop-messages to TRUE. If interval is set to 0, every dropped packet will result in a drop message being posted.

Flags : Read / Write

Default value : 200

Since : 1.18

drop-on-latency

“drop-on-latency” gboolean

Drop oldest buffers when the queue is completely filled.

Flags : Read / Write

Default value : false

faststart-min-packets

“faststart-min-packets” guint

The number of consecutive packets needed to start (set to 0 to disable faststart. The jitterbuffer will by default start after the latency has elapsed)

Flags : Read / Write

Default value : 0

Since : 1.14

latency

“latency” guint

The maximum latency of the jitterbuffer. Packets will be kept in the buffer for at most this time.

Flags : Read / Write

Default value : 200

max-dropout-time

“max-dropout-time” guint

The maximum time (milliseconds) of missing packets tolerated.

Flags : Read / Write

Default value : 60000

max-misorder-time

“max-misorder-time” guint

The maximum time (milliseconds) of misordered packets tolerated.

Flags : Read / Write

Default value : 2000

max-rtcp-rtp-time-diff

“max-rtcp-rtp-time-diff” gint

The maximum amount of time in ms that the RTP time in the RTCP SRs is allowed to be ahead of the last RTP packet we received. Use -1 to disable ignoring of RTCP packets.

Flags : Read / Write

Default value : -1

Since : 1.8

max-ts-offset-adjustment

“max-ts-offset-adjustment” guint64

The maximum number of nanoseconds per frame that time offset may be adjusted with. This is used to avoid sudden large changes to time stamps.

Flags : Read / Write

Default value : 0

min-sync-interval

“min-sync-interval” guint

Determines how often to sync streams using RTCP data or inband NTP-64 header extensions, even if no new RTCP SR information is available.

Flags : Read / Write

Default value : 0

Since : 1.26

mode

“mode” RTPJitterBufferMode *

Control the buffering and timestamping mode used by the jitterbuffer.

Flags : Read / Write

Default value : slave (1)

percent

“percent” gint

The percent of the jitterbuffer that is filled.

Flags : Read

Default value : 0

post-drop-messages

“post-drop-messages” gboolean

Post custom messages to the bus when a packet is dropped by the jitterbuffer due to arriving too late, being already considered lost, or being dropped due to the drop-on-latency property being enabled. Message is of type GST_MESSAGE_ELEMENT and contains a GstStructure named "drop-msg" with the following fields:

guint seqnum: Seqnum of dropped packet.
guint64 timestamp: PTS timestamp of dropped packet.
GString reason: Reason for dropping the packet.
guint num-too-late: Number of packets arriving too late since last drop message.
guint num-drop-on-latency: Number of packets dropped due to the drop-on-latency property since last drop message.

Flags : Read / Write

Default value : false

Since : 1.18

rfc7273-reference-timestamp-meta-only

“rfc7273-reference-timestamp-meta-only” gboolean

When enabled, the jitterbuffer calculates the PTS of the outgoing buffers according to the configured mode as if not RFC7273 mode is enabled.

The timestamps calculated from the RFC7273 clock are only put into the reference timestamp meta, if enabled via the corresponding property.

This is useful in combination with the rfc7273-use-system-clock, or generally if synchronization should not be affected by the original sender timestamps but the original sender timestamps should nonetheless be available as metadata.

Flags : Read / Write

Default value : false

Since : 1.24

rfc7273-sync

“rfc7273-sync” gboolean

Synchronize received streams to the RFC7273 clock (requires clock and offset to be provided)

Flags : Read / Write

Default value : false

rfc7273-use-system-clock

“rfc7273-use-system-clock” gboolean

Uses the system clock as media clock in RFC7273 mode instead of instantiating an NTP or PTP clock.

This will always provide the correct sender timestamps in the GstReferenceTimestampMeta as long as the system clock is synced to the actual media clock with at most a few seconds difference.

PTS on outgoing buffers would be as accurate as the synchronization between the actual media clock and the system clock.

This can be useful if only recovery of the original sender timestamps is needed and syncing to a PTP/NTP clock would be unnecessarily complex, or if the system clock already is synchronized to the correct clock and doing it another time inside GStreamer would be unnecessary.

Flags : Read / Write

Default value : false

Since : 1.24

rtx-deadline

“rtx-deadline” gint

The deadline for a valid RTX request in ms.

How long the RTX RTCP will be valid for. When -1 is used, the size of the jitterbuffer will be used.

Flags : Read / Write

Default value : -1

Since : 1.10

rtx-delay

“rtx-delay” gint

When a packet did not arrive at the expected time, wait this extra amount of time before sending a retransmission event.

When -1 is used, the max jitter will be used as extra delay.

Flags : Read / Write

Default value : -1

Since : 1.2

rtx-delay-reorder

“rtx-delay-reorder” gint

Assume that a retransmission event should be sent when we see this much packet reordering.

When -1 is used, the value will be estimated based on observed packet reordering. When 0 is used packet reordering alone will not cause a retransmission event (Since 1.10).

Flags : Read / Write

Default value : 3

Since : 1.2

rtx-max-retries

“rtx-max-retries” gint

The maximum number of retries to request a retransmission.

This implies that as maximum (rtx-max-retries + 1) retransmissions will be requested. When -1 is used, the number of retransmission request will not be limited.

Flags : Read / Write

Default value : -1

Since : 1.6

rtx-min-delay

“rtx-min-delay” guint

When a packet did not arrive at the expected time, wait at least this extra amount of time before sending a retransmission event.

Flags : Read / Write

Default value : 0

Since : 1.6

rtx-min-retry-timeout

“rtx-min-retry-timeout” gint

The minimum amount of time between retry timeouts. When GstRtpJitterBuffer::rtx-retry-timeout is -1, this value ensures a minimum interval between retry timeouts.

When -1 is used, the value will be estimated based on the packet spacing.

Flags : Read / Write

Default value : -1

Since : 1.6

rtx-next-seqnum

“rtx-next-seqnum” gboolean

Estimate when the next packet should arrive and schedule a retransmission request for it. This is, when packet N arrives, a GstRTPRetransmission event is schedule for packet N+1. So it will be requested if it does not arrive at the expected time. The expected time is calculated using the dts of N and the packet spacing.

Flags : Read / Write

Default value : true

Since : 1.6

rtx-retry-period

“rtx-retry-period” gint

The amount of time to try to get a retransmission.

When -1 is used, the value will be estimated based on the jitterbuffer latency and the observed round trip time.

Flags : Read / Write

Default value : -1

Since : 1.2

rtx-retry-timeout

“rtx-retry-timeout” gint

When no packet has been received after sending a retransmission event for this time, retry sending a retransmission event.

When -1 is used, the value will be estimated based on observed round trip time.

Flags : Read / Write

Default value : -1

Since : 1.2

rtx-stats-timeout

“rtx-stats-timeout” guint

The time to wait for a retransmitted packet after it has been considered lost in order to collect RTX statistics.

Flags : Read / Write

Default value : 1000

Since : 1.10

stats

“stats” GstStructure *

Various jitterbuffer statistics. This property returns a GstStructure with name application/x-rtp-jitterbuffer-stats with the following fields:

guint64 num-pushed: the number of packets pushed out.
guint64 num-lost: the number of packets considered lost.
guint64 num-late: the number of packets arriving too late.
guint64 num-duplicates: the number of duplicate packets.
guint64 avg-jitter: the average jitter in nanoseconds.
guint64 rtx-count: the number of retransmissions requested.
guint64 rtx-success-count: the number of successful retransmissions.
gdouble rtx-per-packet: average number of RTX per packet.
guint64 rtx-rtt: average round trip time per RTX.

Flags : Read

Default value :

application/x-rtp-jitterbuffer-stats, num-pushed=(guint64)0, num-lost=(guint64)0, num-late=(guint64)0, num-duplicates=(guint64)0, avg-jitter=(guint64)0, rtx-count=(guint64)0, rtx-success-count=(guint64)0, rtx-per-packet=(double)0, rtx-rtt=(guint64)0;

Since : 1.4

sync-interval

“sync-interval” guint

Determines how often to sync streams using RTCP data or inband NTP-64 header extensions.

Flags : Read / Write

Default value : 0

Since : 1.22

ts-offset

“ts-offset” gint64

Adjust GStreamer output buffer timestamps in the jitterbuffer with offset. This is mainly used to ensure interstream synchronisation.

Flags : Read / Write

Default value : 0

Named constants

RTPJitterBufferMode

The different buffer modes for a jitterbuffer.

Members

none (0) – Only use RTP timestamps

slave (1) – Slave receiver to sender clock

buffer (2) – Do low/high watermark buffering

synced (4) – Synchronized sender and receiver clocks

The results of the search are