Multiple gstreamer aravissrc instances hang

Using Aravis
1

This gstreamer pipeline works as expected (cam0):

gst-launch-1.0 aravissrc camera-name="cam0 " ! video/x-raw,format=GRAY8,width=2448,height=2048 ! videoconvert ! queue ! x264enc speed-preset=ultrafast ! rtspclientsink location=rtsp://localhost:8554/cam0

as does this (cam1):

gst-launch-1.0 aravissrc camera-name="cam1" ! video/x-raw,format=GRAY8,width=2448,height=2048 ! videoconvert ! queue ! x264enc speed-preset=ultrafast ! rtspclientsink location=rtsp://localhost:8554/cam1

However, if one of them is already running (e.g., cam0) and I start the second (e.g., cam1), then the first instance hangs and eventually times out with this error:

ERROR: from element /GstPipeline:pipeline0/GstAravis:aravis0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstAravis:aravis0:
streaming stopped, reason error (-5)
Execution ended after 0:00:40.284995615
Setting pipeline to NULL ...
Freeing pipeline ...

Meanwhile, the second instance (e.g., cam1) hangs at one of the “Redistribute latency…” lines and eventually exits (after 5-6 minutes) with:

ERROR: from element /GstPipeline:pipeline0/GstAravis:aravis0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstAravis:aravis0:
streaming stopped, reason error (-5)
Execution ended after 0:06:09.875235590
Setting pipeline to NULL ...
Freeing pipeline ...

I’ve seen posts from at least one person claiming to be using multiple gstreamer aravissrc instances. Any ideas why this doesn’t work for me? Thanks!

PS. Before finding aravissrc, I was using python to drive a gstreamer pipeline (without aravissrc) and I had no problem with two instances of that script running.

PPS. I also tried simpler pipelines, with the same result (with and without the videoconvert stage):

gst-launch-1.0 aravissrc camera-name="cam0" ! videoconvert ! fakesink
gst-launch-1.0 aravissrc camera-name="cam1" ! videoconvert ! fakesink
2

Hi,

Multiple instances of aravissrc is supposed to work. A possible reason it does not for you would be an excessive load on your machine. You should try to increase the debug output of gstreamer using GST_DEBUG environment variable.

https://gstreamer.freedesktop.org/documentation/tutorials/basic/debugging-tools.html?gi-language=c

3

Thanks for the debugging pointer.

For now, I have worked around the problem by adding packet-delay= to aravissrc in the gstreamer pipeline:

gst-launch-1.0 aravissrc camera-name="$NAME" num-arv-buffers=100 packet-delay=15000 !

I started with 10000 and have been increasing it as the cameras have issues. I’m not sure why this is helpful, since the machine has enough headroom in terms of CPU (E3-1245 v5 @ 3.50GHz) and RAM (64GB) and these cameras are on a dedicated 1gbe port and the traffic is hardly saturating the link.

If the instability continues, I’ll collect more debugging data and followup here.

4

I started with 10000 and have been increasing it as the cameras have issues. I’m not sure why this is helpful, since the machine has enough headroom in terms of CPU (E3-1245 v5 @ 3.50GHz) and RAM (64GB) and these cameras are on a dedicated 1gbe port and the traffic is hardly saturating the link.

You may have have plenty of CPU power and a large enough ethernet bandwitdh, you can still have transmission issues.

Depending on the model of cameras, when a buffer is ready, it can be emitted full bandwidth. If you have several 1GB/s cameras doing the same thing at the same time, going through a 1GB/s switch, packets must be stored in the switch buffers, which may be full at some point if the acquired images are big. That is where the packet delay can help, by limiting the packet rate of each cameras, avoiding the buffer full issue.

You can check the statistics of aravis packet transmission by enabling the debug output using ARV_DEBUG:

ARV_DEBUG=stream:3 gst-launch-1.0 aravissrc ...