[SOLVED] Crash in GStreamer qmlglsink pipeline dynamically rebind to different GstGLVideoItem

Issue

I’ve used one of existing qmlglsink examples to stream video feed from 4 IP Cameras.
4 Pipelines are created before engine load.

for(int i = 0; i < maxCameras; ++i) {
    GstElement* pipeline = gst_pipeline_new (NULL);
    GstElement* src = gst_element_factory_make ("udpsrc", NULL);
    GstElement* parse = gst_element_factory_make ("jpegparse", NULL);
    GstElement* decoder = gst_element_factory_make ("jpegdec", NULL);
    GstElement* glcolorconvert = gst_element_factory_make ("glcolorconvert", NULL);
    GstElement* glupload = gst_element_factory_make ("glupload", NULL);
    GstElement *sink = gst_element_factory_make ("qmlglsink", NULL);

    g_assert (src && parse && decoder && glupload && glcolorconvert && sink);

    g_object_set (G_OBJECT (src), "port", startingPort + i, NULL);

    g_object_set (G_OBJECT (sink), "sync", FALSE, NULL);

    gst_bin_add_many (GST_BIN (pipeline), src, parse, decoder, glupload, glcolorconvert, sink, NULL);
    if (!gst_element_link_many ( src, parse, decoder, glupload, glcolorconvert, sink, NULL)) {
        qDebug() << "Linking GStreamer pipeline elements failed";
    }

    sinks.insert(std::make_pair(QString::number(startingPort+i), sink));
    pipelines.insert(std::make_pair(QString::number(startingPort+i), pipeline));
}

In Qml sink is connected and processed with

import QtQuick 2.15
import QtQuick.Layouts 1.15
import CustomProject 1.0
import org.freedesktop.gstreamer.GLVideoItem 1.0

Item {
    id: root

    signal clicked()
    required property int udpPort
    property var camConnect: undefined
    onUdpPortChanged: { setupConnection(); }

    onVisibleChanged: {
        if (visible) { setupConnection();
        } else { camConnect = undefined }
    }

    GstGLVideoItem {
        id: videoItem
        anchors.fill: parent
        function connect() {
            CameraSinksFactory.connectSink(this, udpPort)
        }
    }
    MouseArea {
        anchors.fill: parent
        onClicked: {
            CameraSinksFactory.stopPipeline(udpPort)
            root.clicked()
        }
    }

    function setupConnection() {
        if (udpPort <= 0 || !root.visible) return;
        videoItem.connect()
        root.camConnect = CameraSinksFactory.getCamConnection(udpPort);
        root.camConnect.resolutionX =// - 15 root.width
        root.camConnect.resolutionY = root.height
        root.camConnect.bitrate = 15000000
        root.camConnect.streaming = root.visible
        CameraSinksFactory.startPipeline(udpPort)
    }
}

Problem: Main screen display 4 (2×2 grid) items using Model (which provides udpPort as unique ID). When User clicks on one Item – feed from this camera should fill whole screen. In Examples they create GridLayout with 4/6 explicit items and just manipulate their visiblity (in effect clicked item is only one remaining and take whole screen).
In my case – I’m using separate Item for full screen view. So I’m disabling streaming (CamConnection class communicating with Cameras and sending commands) and hide GridView. New GstGLVideoItem binds to qmlglsink in pipeline.
Everything is OK, until I repeat click sequence (back to GridView and to fullview). Every time it ends with:

Bail out! ERROR:../ext/qt/gstqsgtexture.cc:134:virtual void
GstQSGTexture::bind(): code should not be reached

** (KMS:20495): CRITICAL **: 15:47:36.937: gst_video_frame_map_id: assertion ‘info->width <= meta->width’ failed
** ERROR:../ext/qt/gstqsgtexture.cc:134:virtual void GstQSGTexture::bind(): code should not be reached

From plugins code analysis it’s happening when INFO (image size read from CAPS) is bigger then size in metadata in provided buffer. Which is understandable – buffer is too small.
I used GST_DEBUG=4/5/6/7 and logs confirm that autodetected caps are matching request in commands sent to camera.
I can use example, but project assumes another panel with those cameras – so above problem will hit me in near future.

How to make this whole setup working? How to rebind pipeline qmlglsink to new QML VideoItem safely?

Solution

Two possible solutions:

• set gst_element_set_state (pipeline, GST_STATE_NULL);, change sink widget to new item and start pipeline gst_element_set_state (pipeline, GST_STATE_PLAYING);
• use Qt 5 MediaPlayer with gst-pipeline as source. When visible set source and execute start(). When not visible reset source to empty (important) and execute stop().

In general – possible benefits from NOT creating pipeline each time are not worth hassle when we dynamically assign new QML Item to pipeline sink.

Answered By – T4ng10r

Answer Checked By – Clifford M. (BugsFixing Volunteer)