Frame Rate Drop

Using Aravis
1

Hello, I’ve been having an ongoing issue with the frame rate dropping off while using Aravis. We are using a Specim-FX10 Hyperspectral Camera and the host machine is an NVIDIA Jetson AGX Orin Development Kit, running Jetson Linux 35.2.1 and Jetpack 5.0.2. Before this, we were using a Jetson Xavier NX running Jetpack 4.6 and had the same issues.

To describe the issue in more detail, basically the frame rate of the camera is supposed to be up to 50fps. The manufacturer created a software tool to use the camera, but it only works on Windows machines. Using that program, we are seeing the full 50fps, with no problems. But using Aravis, we see the frame rate bounce around depending on the situation. When using the arv-camera-test-0.8 tool, it usually bounces around from 50fps, all the way down to 30fps and it will change every second. We also have a custom application that uses the Aravis API and the frame rate there will go from 50fps, all the way to 0 and once it hits 0, it just stays at 0, as if the camera was killed (but I have not seen any error emmitted).

I would appreciate any direction into how I could even begin to debug this issue.

Thanks,
Will

2

Hi Will,

On what machine are you running Windows ?

Did you try to run arv-camera-test-0.8 with -d all ?

Emmanuel.

3

Hi Emmanuel,

Thank you for your swift response. The Windows machine is a Dell XPS 15 7590 running Windows 10. Intel i7-9750H CPU @ 2.6GHz, 6 Cores, 12 Logical Processors. 16GB RAM.

Also, here is the raw output when I run arv-camera-test-0.8 -d all for a few seconds:

Looking for the first available camera
[12:16:35.740] 🅸 interface> Found 0 USB3Vision device (among 6 USB devices)
[12:16:35.741] 🅸 interface> [GvDiscoverSocket::new] Add interface 127.0.0.1 (127.0.0.1)
[12:16:35.741] 🅸 interface> [GvDiscoverSocket::new] Add interface 10.1.10.6 (10.1.10.255)
[12:16:35.741] 🅸 interface> [GvDiscoverSocket::new] Add interface 172.17.0.1 (172.17.255.255)
[12:16:35.741] 🅸 interface> [GvDiscoverSocket::new] Add interface 169.254.157.154 (169.254.255.255)
[12:16:35.818] 🅸 interface> [GvInterface::discovery] Device 'Specim-FX10e-077000051616' found (interface 169.254.157.154) user_id 'specim_fx10' - MAC '70:f8:e7:b0:05:a7'
[12:16:35.818] 🅸 device> [GvDevice::new] Interface address = 169.254.157.154
[12:16:35.818] 🅸 device> [GvDevice::new] Device address = 169.254.3.3
[12:16:35.819] 🅸 device> [GvDevice::load_genicam] xml url = 'Local:FX10e.zip;400000;bbd1' at 0x200
[12:16:35.819] 🅸 device> [GvDevice::load_genicam] Xml address = 0x400000 - size = 0xbbd1 - FX10e.zip
[12:16:36.378] 🅸 device> [GvDevice::load_genicam] Zipped xml data
[12:16:36.459] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevCurrentIPConfigurationLLA'
[12:16:36.459] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevCurrentIPConfigurationDHCP'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevCurrentIPConfigurationPersistentIP'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevCurrentIPAddress'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevCurrentSubnetMask'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevCurrentDefaultGateway'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevNumberOfNetworkInterfaces'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevPersistentIPAddress'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevPersistentSubnetMask'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevPersistentDefaultGateway'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevStreamChannelCount'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevStreamChannelSelector'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCPAddrCalc'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevTimestampTickFrequency'
[12:16:36.460] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCPHostPort'
[12:16:36.461] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCPSFireTestPacket'
[12:16:36.461] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCPSDoNotFragment'
[12:16:36.461] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCPSBigEndian'
[12:16:36.461] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCPSPacketSize'
[12:16:36.461] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCPD'
[12:16:36.461] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCDA'
[12:16:36.461] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCSP'
[12:16:36.461] 🅸 genicam> [Gc::set_default_node_data] Add 'ArvGevSCCFGMultipart'
[12:16:36.462] 🅸 device> [GvDevice::new] Device endianness = big
[12:16:36.462] 🅸 device> [GvDevice::new] Packet resend     = yes
[12:16:36.462] 🅸 device> [GvDevice::new] Write memory      = yes
[12:16:36.462] 🅸 device> [GvDevice::new] Legacy endianness handling = no
vendor name            = Specim
model name             = FX10e
device serial number   = 077000051616
image width            = 1024
image height           = 224
horizontal binning     = 1
vertical binning       = 2
exposure               = 19780 µs
gain                   = 1 dB
payload                = 344064 bytes
gv n_stream channels   = 1
gv current channel     = 0
gv packet delay        = 0 ns
[12:16:36.466] 🅸 device> [GvDevice::create_stream] Number of stream channels = 1
[12:16:36.468] 🅸 device> [GvDevice::auto_packet_size] Current packet size check successfull (1060 bytes)
[12:16:36.468] 🅸 stream> [GvStream::stream_new] Stream channel = 0
[12:16:36.469] 🅸 stream> [GvStream::stream_new] Packet size = 1060 byte(s)
[12:16:36.470] 🅸 stream> [GvStream::stream_new] Destination stream port = 43087
[12:16:36.470] 🅸 stream> [GvStream::stream_new] Source stream port = 0
[12:16:36.470] 🅸 stream> [GvStream::loop] Standard socket method
gv packet size         = 1060 bytes
 50 frames/s -    17.2 MiB/s
[12:16:38.343] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40605, packet id = 229 frame id = 92
[12:16:38.344] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       92 
                     packet_id    =      229
                     data_size    =        0
[12:16:38.344] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 42978, packet id = 269 frame id = 92
[12:16:38.345] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       92 
                     packet_id    =      269
                     data_size    =        0
[12:16:38.382] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 40910, n_packet_requests = 212 (338 packets/frame), frame_id = 94
[12:16:38.383] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 60792, packet id = 253 frame id = 93
[12:16:38.383] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       93 
                     packet_id    =      253
                     data_size    =        0
[12:16:38.384] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 60792, packet id = 291 frame id = 93
[12:16:38.384] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       93 
                     packet_id    =      291
                     data_size    =        0
[12:16:38.384] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43344, packet id = 23 frame id = 94
[12:16:38.384] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       94 
                     packet_id    =       23
                     data_size    =        0
[12:16:38.422] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41135, n_packet_requests = 195 (338 packets/frame), frame_id = 96
[12:16:38.423] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61021, packet id = 335 frame id = 95
[12:16:38.423] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       95 
                     packet_id    =      335
                     data_size    =        0
[12:16:38.424] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41135, packet id = 21 frame id = 96
[12:16:38.424] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       96 
                     packet_id    =       21
                     data_size    =        0
[12:16:38.424] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43402, packet id = 27 frame id = 96
[12:16:38.424] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       96 
                     packet_id    =       27
                     data_size    =        0
[12:16:38.425] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43812, packet id = 104 frame id = 96
[12:16:38.425] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       96 
                     packet_id    =      104
                     data_size    =        0
[12:16:38.461] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 92 at dt = 159048
[12:16:38.462] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41266, n_packet_requests = 170 (338 packets/frame), frame_id = 98
 41 frames/s -    14.1 MiB/s - 1 error
[12:16:38.481] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40657, packet id = 150 frame id = 99
[12:16:38.482] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       99 
                     packet_id    =      150
                     data_size    =        0
[12:16:38.482] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40657, packet id = 181 frame id = 99
[12:16:38.482] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       99 
                     packet_id    =      181
                     data_size    =        0
[12:16:38.483] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 93 at dt = 161747
[12:16:38.483] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 42419, packet id = 225 frame id = 99
[12:16:38.483] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       99 
                     packet_id    =      225
                     data_size    =        0
[12:16:38.484] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43209, packet id = 291 frame id = 99
[12:16:38.484] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =       99 
                     packet_id    =      291
                     data_size    =        0
[12:16:38.484] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 23596, n_packet_requests = 144 (338 packets/frame), frame_id = 100
[12:16:38.484] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 94 at dt = 143370
[12:16:38.521] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40683, packet id = 21 frame id = 101
[12:16:38.522] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      101 
                     packet_id    =       21
                     data_size    =        0
[12:16:38.522] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40683, packet id = 28 frame id = 101
[12:16:38.523] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      101 
                     packet_id    =       28
                     data_size    =        0
[12:16:38.524] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 95 at dt = 162491
[12:16:38.524] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43607, packet id = 51 frame id = 101
[12:16:38.524] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      101 
                     packet_id    =       51
                     data_size    =        0
[12:16:38.525] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 96 at dt = 144064
[12:16:38.541] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 98 at dt = 119353
[12:16:38.562] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41271, n_packet_requests = 208 (338 packets/frame), frame_id = 103
[12:16:38.562] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61151, packet id = 227 frame id = 102
[12:16:38.563] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      102 
                     packet_id    =      227
                     data_size    =        0
[12:16:38.563] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 63035, packet id = 269 frame id = 102
[12:16:38.563] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      102 
                     packet_id    =      269
                     data_size    =        0
[12:16:38.563] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 23571, n_packet_requests = 97 (338 packets/frame), frame_id = 104
[12:16:38.564] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43480, packet id = 21 frame id = 103
[12:16:38.564] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      103 
                     packet_id    =       21
                     data_size    =        0
[12:16:38.564] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43869, packet id = 28 frame id = 103
[12:16:38.564] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      103 
                     packet_id    =       28
                     data_size    =        0
[12:16:38.600] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 99 at dt = 159120
[12:16:38.600] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 100 at dt = 139235
[12:16:38.602] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41642, n_packet_requests = 154 (338 packets/frame), frame_id = 105
[12:16:38.620] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40355, packet id = 150 frame id = 106
[12:16:38.620] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      106 
                     packet_id    =      150
                     data_size    =        0
[12:16:38.620] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40355, packet id = 181 frame id = 106
[12:16:38.620] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      106 
                     packet_id    =      181
                     data_size    =        0
[12:16:38.621] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40355, packet id = 251 frame id = 106
[12:16:38.621] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      106 
                     packet_id    =      251
                     data_size    =        0
[12:16:38.639] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 101 at dt = 159143
[12:16:38.640] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 39797, n_packet_requests = 233 (338 packets/frame), frame_id = 107
[12:16:38.660] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40094, packet id = 191 frame id = 108
[12:16:38.660] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      108 
                     packet_id    =      191
                     data_size    =        0
[12:16:38.680] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 102 at dt = 179340
[12:16:38.680] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 103 at dt = 159460
[12:16:38.680] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 104 at dt = 139551
[12:16:38.680] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 105 at dt = 119678
[12:16:38.720] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 106 at dt = 140358
[12:16:38.720] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 107 at dt = 120464
[12:16:38.760] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 108 at dt = 140131
[12:16:38.999] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40789, packet id = 203 frame id = 125
[12:16:39.000] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      125 
                     packet_id    =      203
                     data_size    =        0
[12:16:39.001] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40789, packet id = 247 frame id = 125
[12:16:39.001] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      125 
                     packet_id    =      247
                     data_size    =        0
[12:16:39.001] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43469, packet id = 291 frame id = 125
[12:16:39.001] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      125 
                     packet_id    =      291
                     data_size    =        0
[12:16:39.039] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41088, n_packet_requests = 198 (338 packets/frame), frame_id = 127
[12:16:39.039] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 60890, packet id = 335 frame id = 126
[12:16:39.040] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      126 
                     packet_id    =      335
                     data_size    =        0
[12:16:39.040] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41088, packet id = 21 frame id = 127
[12:16:39.040] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      127 
                     packet_id    =       21
                     data_size    =        0
[12:16:39.040] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43079, packet id = 27 frame id = 127
[12:16:39.040] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      127 
                     packet_id    =       27
                     data_size    =        0
[12:16:39.041] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43608, packet id = 104 frame id = 127
[12:16:39.041] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      127 
                     packet_id    =      104
                     data_size    =        0
[12:16:39.078] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 40783, n_packet_requests = 172 (338 packets/frame), frame_id = 129
[12:16:39.079] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61625, packet id = 262 frame id = 128
[12:16:39.079] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      128 
                     packet_id    =      262
                     data_size    =        0
[12:16:39.080] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41740, packet id = 20 frame id = 129
[12:16:39.080] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      129 
                     packet_id    =       20
                     data_size    =        0
[12:16:39.080] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43113, packet id = 26 frame id = 129
[12:16:39.080] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      129 
                     packet_id    =       26
                     data_size    =        0
[12:16:39.081] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43639, packet id = 33 frame id = 129
[12:16:39.081] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      129 
                     packet_id    =       33
                     data_size    =        0
[12:16:39.101] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 125 at dt = 143531
[12:16:39.117] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 39796, n_packet_requests = 89 (338 packets/frame), frame_id = 131
[12:16:39.118] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61300, packet id = 253 frame id = 130
[12:16:39.119] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      130 
                     packet_id    =      253
                     data_size    =        0
[12:16:39.120] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 42621, packet id = 19 frame id = 131
[12:16:39.120] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      131 
                     packet_id    =       19
                     data_size    =        0
[12:16:39.120] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43241, packet id = 25 frame id = 131
[12:16:39.120] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      131 
                     packet_id    =       25
                     data_size    =        0
[12:16:39.138] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40641, packet id = 229 frame id = 132
[12:16:39.138] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      132 
                     packet_id    =      229
                     data_size    =        0
[12:16:39.139] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40641, packet id = 269 frame id = 132
[12:16:39.139] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      132 
                     packet_id    =      269
                     data_size    =        0
[12:16:39.140] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 126 at dt = 162073
[12:16:39.156] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 127 at dt = 159113
[12:16:39.158] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41040, n_packet_requests = 133 (338 packets/frame), frame_id = 133
[12:16:39.160] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 42912, packet id = 22 frame id = 133
[12:16:39.160] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      133 
                     packet_id    =       22
                     data_size    =        0
[12:16:39.160] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43246, packet id = 53 frame id = 133
[12:16:39.160] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      133 
                     packet_id    =       53
                     data_size    =        0
[12:16:39.179] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 128 at dt = 161697
[12:16:39.196] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 129 at dt = 159112
[12:16:39.197] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 59659, n_packet_requests = 209 (338 packets/frame), frame_id = 134
[12:16:39.198] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 39773, n_packet_requests = 120 (338 packets/frame), frame_id = 135
[12:16:39.217] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40708, packet id = 150 frame id = 136
[12:16:39.218] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      136 
                     packet_id    =      150
                     data_size    =        0
[12:16:39.218] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40708, packet id = 203 frame id = 136
[12:16:39.219] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      136 
                     packet_id    =      203
                     data_size    =        0
[12:16:39.219] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 130 at dt = 162232
[12:16:39.220] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43333, packet id = 247 frame id = 136
[12:16:39.220] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      136 
                     packet_id    =      247
                     data_size    =        0
[12:16:39.220] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 23760, n_packet_requests = 121 (338 packets/frame), frame_id = 137
[12:16:39.220] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 131 at dt = 143291
[12:16:39.238] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 132 at dt = 140825
[12:16:39.257] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 40885, n_packet_requests = 208 (338 packets/frame), frame_id = 138
[12:16:39.258] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40885, packet id = 21 frame id = 138
[12:16:39.258] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      138 
                     packet_id    =       21
                     data_size    =        0
[12:16:39.259] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40885, packet id = 27 frame id = 138
[12:16:39.259] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      138 
                     packet_id    =       27
                     data_size    =        0
[12:16:39.259] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43358, packet id = 52 frame id = 138
[12:16:39.260] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      138 
                     packet_id    =       52
                     data_size    =        0
[12:16:39.261] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 133 at dt = 143600
[12:16:39.261] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 134 at dt = 124615
[12:16:39.261] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 135 at dt = 104729
[12:16:39.297] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41096, n_packet_requests = 195 (338 packets/frame), frame_id = 140
[12:16:39.298] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 62124, packet id = 226 frame id = 139
[12:16:39.299] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      139 
                     packet_id    =      226
                     data_size    =        0
[12:16:39.299] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 63351, packet id = 291 frame id = 139
[12:16:39.299] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      139 
                     packet_id    =      291
                     data_size    =        0
[12:16:39.301] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 4768, n_packet_requests = 147 (338 packets/frame), frame_id = 142
[12:16:39.336] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 136 at dt = 159108
[12:16:39.336] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 137 at dt = 139221
[12:16:39.375] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 138 at dt = 159110
[12:16:39.379] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 62938, packet id = 313 frame id = 143
[12:16:39.379] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      143 
                     packet_id    =      313
                     data_size    =        0
[12:16:39.398] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 62531, packet id = 102 frame id = 144
[12:16:39.398] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      144 
                     packet_id    =      102
                     data_size    =        0
[12:16:39.398] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 63040, packet id = 203 frame id = 144
[12:16:39.399] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      144 
                     packet_id    =      203
                     data_size    =        0
[12:16:39.399] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 63040, packet id = 247 frame id = 144
[12:16:39.399] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      144 
                     packet_id    =      247
                     data_size    =        0
[12:16:39.399] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 139 at dt = 163354
[12:16:39.399] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 140 at dt = 143479
[12:16:39.415] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 142 at dt = 119286
[12:16:39.416] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 39771, n_packet_requests = 131 (338 packets/frame), frame_id = 146
[12:16:39.418] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 62102, packet id = 305 frame id = 145
[12:16:39.418] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      145 
                     packet_id    =      305
                     data_size    =        0
[12:16:39.418] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 62961, packet id = 335 frame id = 145
[12:16:39.418] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      145 
                     packet_id    =      335
                     data_size    =        0
[12:16:39.436] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 39773, n_packet_requests = 85 (338 packets/frame), frame_id = 147
[12:16:39.437] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41255, packet id = 104 frame id = 147
[12:16:39.437] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      147 
                     packet_id    =      104
                     data_size    =        0
[12:16:39.438] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41255, packet id = 159 frame id = 147
[12:16:39.438] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      147 
                     packet_id    =      159
                     data_size    =        0
[12:16:39.457] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41100, packet id = 90 frame id = 148
[12:16:39.457] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      148 
                     packet_id    =       90
                     data_size    =        0
[12:16:39.458] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41100, packet id = 104 frame id = 148
[12:16:39.458] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      148 
                     packet_id    =      104
                     data_size    =        0
[12:16:39.458] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43403, packet id = 205 frame id = 148
[12:16:39.459] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      148 
                     packet_id    =      205
                     data_size    =        0
[12:16:39.460] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 24954, n_packet_requests = 172 (338 packets/frame), frame_id = 149
 18 frames/s -    6.19 MiB/s - 32 errors
[12:16:39.479] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 143 at dt = 163301
[12:16:39.515] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 144 at dt = 178993
[12:16:39.515] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 59661, n_packet_requests = 173 (338 packets/frame), frame_id = 150
[12:16:39.519] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 145 at dt = 163348
[12:16:39.519] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 146 at dt = 143461
[12:16:39.537] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 147 at dt = 141442
[12:16:39.537] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61899, packet id = 128 frame id = 151
[12:16:39.537] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      151 
                     packet_id    =      128
                     data_size    =        0
[12:16:39.538] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 63010, packet id = 194 frame id = 151
[12:16:39.538] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      151 
                     packet_id    =      194
                     data_size    =        0
[12:16:39.538] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 23802, n_packet_requests = 122 (338 packets/frame), frame_id = 153
[12:16:39.555] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 60350, packet id = 335 frame id = 152
[12:16:39.556] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      152 
                     packet_id    =      335
                     data_size    =        0
[12:16:39.574] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 148 at dt = 159114
[12:16:39.575] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 149 at dt = 139216
[12:16:39.578] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 23703, n_packet_requests = 140 (338 packets/frame), frame_id = 155
[12:16:39.594] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 150 at dt = 139224
[12:16:39.597] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 63065, packet id = 70 frame id = 154
[12:16:39.598] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      154 
                     packet_id    =       70
                     data_size    =        0
[12:16:39.598] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 23847, n_packet_requests = 185 (338 packets/frame), frame_id = 156
[12:16:39.634] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 39774, n_packet_requests = 101 (338 packets/frame), frame_id = 157
[12:16:39.635] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40826, packet id = 98 frame id = 157
[12:16:39.635] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      157 
                     packet_id    =       98
                     data_size    =        0
[12:16:39.654] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 151 at dt = 179005
[12:16:39.656] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 152 at dt = 160811
[12:16:39.656] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 153 at dt = 140924
[12:16:39.657] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43119, packet id = 150 frame id = 158
[12:16:39.657] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      158 
                     packet_id    =      150
                     data_size    =        0
[12:16:39.658] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 23574, n_packet_requests = 151 (338 packets/frame), frame_id = 159
[12:16:39.658] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43458, packet id = 181 frame id = 158
[12:16:39.658] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      158 
                     packet_id    =      181
                     data_size    =        0
[12:16:39.658] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43458, packet id = 251 frame id = 158
[12:16:39.658] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      158 
                     packet_id    =      251
                     data_size    =        0
[12:16:39.695] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40652, packet id = 104 frame id = 160
[12:16:39.695] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      160 
                     packet_id    =      104
                     data_size    =        0
[12:16:39.698] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 154 at dt = 163349
[12:16:39.698] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 155 at dt = 143454
[12:16:39.698] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 156 at dt = 123574
[12:16:39.734] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 39781, n_packet_requests = 141 (338 packets/frame), frame_id = 162
[12:16:39.735] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 60904, packet id = 291 frame id = 161
[12:16:39.736] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      161 
                     packet_id    =      291
                     data_size    =        0
[12:16:39.736] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 157 at dt = 142047
[12:16:39.755] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41691, packet id = 150 frame id = 163
[12:16:39.756] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      163 
                     packet_id    =      150
                     data_size    =        0
[12:16:39.756] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 42795, packet id = 181 frame id = 163
[12:16:39.756] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      163 
                     packet_id    =      181
                     data_size    =        0
[12:16:39.756] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 42795, packet id = 251 frame id = 163
[12:16:39.757] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      163 
                     packet_id    =      251
                     data_size    =        0
[12:16:39.758] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 158 at dt = 143504
[12:16:39.758] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 24600, n_packet_requests = 179 (338 packets/frame), frame_id = 164
[12:16:39.759] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 159 at dt = 125216
[12:16:39.793] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40133, packet id = 70 frame id = 165
[12:16:39.794] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      165 
                     packet_id    =       70
                     data_size    =        0
[12:16:39.794] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 160 at dt = 140659
[12:16:39.813] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40095, packet id = 251 frame id = 166
[12:16:39.813] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      166 
                     packet_id    =      251
                     data_size    =        0
[12:16:39.835] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 161 at dt = 161480
[12:16:39.835] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 162 at dt = 141597
[12:16:39.873] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 163 at dt = 159673
[12:16:39.873] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 164 at dt = 139790
[12:16:39.893] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 165 at dt = 140172
[12:16:39.913] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 166 at dt = 140200
 35 frames/s -      12 MiB/s - 24 errors
 50 frames/s -    17.2 MiB/s
 50 frames/s -    17.2 MiB/s
 51 frames/s -    17.5 MiB/s
 50 frames/s -    17.2 MiB/s
 50 frames/s -    17.2 MiB/s
[12:16:46.379] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 60886, packet id = 313 frame id = 495
[12:16:46.379] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      495 
                     packet_id    =      313
                     data_size    =        0
[12:16:46.380] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43153, packet id = 102 frame id = 496
[12:16:46.380] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      496 
                     packet_id    =      102
                     data_size    =        0
[12:16:46.380] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43632, packet id = 181 frame id = 496
[12:16:46.380] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      496 
                     packet_id    =      181
                     data_size    =        0
[12:16:46.381] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 44552, packet id = 273 frame id = 496
[12:16:46.381] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      496 
                     packet_id    =      273
                     data_size    =        0
[12:16:46.418] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41131, n_packet_requests = 198 (338 packets/frame), frame_id = 498
[12:16:46.418] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61020, packet id = 228 frame id = 497
[12:16:46.419] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      497 
                     packet_id    =      228
                     data_size    =        0
[12:16:46.419] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 62844, packet id = 269 frame id = 497
[12:16:46.419] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      497 
                     packet_id    =      269
                     data_size    =        0
[12:16:46.420] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 43377, packet id = 22 frame id = 498
[12:16:46.420] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      498 
                     packet_id    =       22
                     data_size    =        0
[12:16:46.421] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 44769, packet id = 105 frame id = 498
[12:16:46.422] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      498 
                     packet_id    =      105
                     data_size    =        0
[12:16:46.458] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 41320, n_packet_requests = 169 (338 packets/frame), frame_id = 500
[12:16:46.458] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61193, packet id = 154 frame id = 499
[12:16:46.459] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      499 
                     packet_id    =      154
                     data_size    =        0
[12:16:46.459] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61193, packet id = 181 frame id = 499
[12:16:46.459] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      499 
                     packet_id    =      181
                     data_size    =        0
[12:16:46.459] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 63342, packet id = 225 frame id = 499
[12:16:46.460] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      499 
                     packet_id    =      225
                     data_size    =        0
[12:16:46.460] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 64319, packet id = 336 frame id = 499
[12:16:46.460] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      499 
                     packet_id    =      336
                     data_size    =        0
[12:16:46.460] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 44446, packet id = 22 frame id = 500
[12:16:46.461] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      500 
                     packet_id    =       22
                     data_size    =        0
[12:16:46.461] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 44978, packet id = 79 frame id = 500
[12:16:46.461] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      500 
                     packet_id    =       79
                     data_size    =        0
[12:16:46.462] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 45777, packet id = 105 frame id = 500
[12:16:46.462] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      500 
                     packet_id    =      105
                     data_size    =        0
[12:16:46.478] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 495 at dt = 160905
 41 frames/s -    14.1 MiB/s - 1 error
[12:16:46.496] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 496 at dt = 159141
[12:16:46.496] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 40879, n_packet_requests = 148 (338 packets/frame), frame_id = 502
[12:16:46.497] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 60752, packet id = 228 frame id = 501
[12:16:46.497] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      501 
                     packet_id    =      228
                     data_size    =        0
[12:16:46.497] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 61186, packet id = 269 frame id = 501
[12:16:46.497] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      501 
                     packet_id    =      269
                     data_size    =        0
[12:16:46.497] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41313, packet id = 20 frame id = 502
[12:16:46.497] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      502 
                     packet_id    =       20
                     data_size    =        0
[12:16:46.497] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41313, packet id = 27 frame id = 502
[12:16:46.497] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      502 
                     packet_id    =       27
                     data_size    =        0
[12:16:46.498] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41992, packet id = 100 frame id = 502
[12:16:46.498] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      502 
                     packet_id    =      100
                     data_size    =        0
[12:16:46.535] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 497 at dt = 178989
[12:16:46.535] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 498 at dt = 159100
[12:16:46.536] 🅸 stream-thread> [GvStream::missing_packet_check] Maximum number of requests reached at dt = 59649, n_packet_requests = 113 (338 packets/frame), frame_id = 503
[12:16:46.536] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40888, packet id = 61 frame id = 504
[12:16:46.537] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      504 
                     packet_id    =       61
                     data_size    =        0
[12:16:46.537] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41239, packet id = 158 frame id = 504
[12:16:46.537] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      504 
                     packet_id    =      158
                     data_size    =        0
[12:16:46.537] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41239, packet id = 187 frame id = 504
[12:16:46.537] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      504 
                     packet_id    =      187
                     data_size    =        0
[12:16:46.537] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 41239, packet id = 253 frame id = 504
[12:16:46.537] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      504 
                     packet_id    =      253
                     data_size    =        0
[12:16:46.575] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 499 at dt = 178957
[12:16:46.575] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 500 at dt = 159084
[12:16:46.576] 🅸 stream-thread> [GvStream::process_packet] Error packet at dt = 40409, packet id = 182 frame id = 506
[12:16:46.576] 🅸 sp> packet_type  = packet-unavailable (0x800c)
                     content_type =  payload (0x0003)
                     frame_id     =      506 
                     packet_id    =      182
                     data_size    =        0
[12:16:46.597] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 501 at dt = 161246
[12:16:46.598] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 502 at dt = 142344
[12:16:46.615] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 503 at dt = 139212
[12:16:46.637] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 504 at dt = 141289
[12:16:46.676] 🆆 stream-thread> [GvStream::check_frame_completion] Timeout for frame 506 at dt = 140452
 48 frames/s -    16.5 MiB/s - 10 errors
 50 frames/s -    17.2 MiB/s
^C 50 frames/s -    17.2 MiB/s
n_completed_buffers    = 586
n_failures             = 68
n_underruns            = 0
n_timeouts             = 68
n_aborted              = 0
n_missing_frames       = 0
n_size_mismatch_errors = 0
n_received_packets     = 212416
n_missing_packets      = 13560
n_error_packets        = 101
n_ignored_packets      = 0
n_resend_requests      = 4562
n_resent_packets       = 1568
n_resend_ratio_reached = 33
n_resend_disabled      = 101
n_duplicated_packets   = 0
n_transferred_bytes    = 217854136
n_ignored_bytes        = 0
[12:16:49.503] 🅸 stream>     bins    ;frame_retent; packet_time;inter_packet
                                    0;           0;      136404;      212010
                                 2000;         566;       73620;          17
                                 4000;           0;        1398;           0
                                 6000;           0;         134;           0
                                 8000;           0;           0;           0
                                10000;           0;           0;           0
                                12000;           0;           0;           0
                                14000;           0;           0;           0
                                16000;           0;           0;           0
                                18000;           0;           0;           0
                                20000;           3;           0;          18
                                22000;           0;           0;           6
                                24000;           0;         842;           0
                                26000;           0;         255;           0
                                28000;           0;           0;           0
                                30000;           0;           0;           0
                                32000;           0;           0;           1
                                34000;           0;           0;          13
                                36000;           0;           0;          26
                                38000;           0;           0;           5
                                40000;           3;          37;           0
                                42000;           0;          29;           0
                                44000;           0;           5;           0
                                46000;           0;           0;           0
                                48000;           0;           0;           0
                                50000;           0;           0;           0
                                52000;           0;           0;           0
                                54000;           0;           0;           0
                                56000;           0;           0;           1
                                58000;           0;           0;           2
                                60000;           3;          16;           0
                                62000;           0;          13;           0
                                64000;           0;           1;           0
                                66000;           0;           0;           0
                                68000;           0;           0;           0
                                70000;           0;           0;           0
                                72000;           0;           0;           0
                                74000;           0;           0;           0
                                76000;           0;           0;           0
                                78000;           1;           0;           0
                                80000;           2;           0;           0
                                82000;           0;           0;           0
                                84000;           0;           0;           0
                                86000;           0;           0;           0
                                88000;           0;           0;           0
                                90000;           0;           0;           0
                                92000;           0;           0;           0
                                94000;           0;           0;           0
                                96000;           0;           0;           0
                                98000;           0;           0;           0
                               100000;           3;           0;           0
                               102000;           0;           0;           0
                               104000;           1;           0;           0
                               106000;           0;           0;           0
                               108000;           0;           0;           0
                               110000;           0;           0;           0
                               112000;           0;           0;           0
                               114000;           0;           0;           0
                               116000;           0;           0;           0
                               118000;           3;           0;           0
                               120000;           5;           0;           0
                               122000;           2;           0;           0
                               124000;           3;           0;           0
                               126000;           0;           0;           0
                               128000;           0;           0;           0
                               130000;           0;           0;           0
                               132000;           0;           0;           0
                               134000;           0;           0;           0
                               136000;           0;           0;           0
                               138000;           7;           0;           0
                               140000;          11;           0;           0
                               142000;          10;           0;           0
                               144000;           1;           0;           0
                               146000;           0;           0;           0
                               148000;           0;           0;           0
                               150000;           0;           0;           0
                               152000;           0;           0;           0
                               154000;           0;           0;           0
                               156000;           0;           0;           0
                               158000;          13;           0;           0
                               160000;           6;           0;           0
                               162000;           7;           0;           0
                               164000;           0;           0;           0
                               166000;           0;           0;           0
                               168000;           0;           0;           0
                               170000;           0;           0;           0
                               172000;           0;           0;           0
                               174000;           0;           0;           0
                               176000;           0;           0;           0
                               178000;           5;           0;           0
                         -------------
                         >=    180000;           0;           0;           0
                         <          0;           0;           0;           0
                         min         :        2035;           0;           0
                         max         :      179340;       64319;       58872
                         last max at :         101;      161156;       44462
                         counter     :         655:      212754:      212099
[12:16:49.504] 🅸 stream> [GvStream::finalize] n_completed_buffers    = 587
[12:16:49.504] 🅸 stream> [GvStream::finalize] n_failures             = 68
[12:16:49.504] 🅸 stream> [GvStream::finalize] n_underruns            = 0
[12:16:49.504] 🅸 stream> [GvStream::finalize] n_timeouts             = 68
[12:16:49.504] 🅸 stream> [GvStream::finalize] n_aborted              = 0
[12:16:49.504] 🅸 stream> [GvStream::finalize] n_missing_frames       = 0
[12:16:49.504] 🅸 stream> [GvStream::finalize] n_size_mismatch_errors = 0
[12:16:49.504] 🅸 stream> [GvStream::finalize] n_received_packets     = 212754
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_missing_packets      = 13560
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_error_packets        = 101
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_ignored_packets      = 0
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_resend_requests      = 4562
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_resent_packets       = 1568
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_resend_ratio_reached = 33
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_resend_disabled      = 101
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_duplicated_packets   = 0
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_transferred_bytes    = 218200948
[12:16:49.505] 🅸 stream> [GvStream::finalize] n_ignored_bytes        = 0
[12:16:49.505] 🅸 stream> [Stream::finalize] Flush 50 buffer[s] in input queue
[12:16:49.505] 🅸 stream> [Stream::finalize] Flush 0 buffer[s] in output queue
4

@Emmanuel Thank you for your help on this. The debug output allowed to me figure out that frames were being dropped because packets were being lost. From here, I did some more research and discovered that this was likely due to the network configurations on my machine. Apparently, the buffers on linux (at least for Jetson’s) are not set high enough by default to support GigE traffic.

For any future visitors, here is my solution for getting my GigE camera to work at 50fps on both my Jetson AGX Orin and Jetson Xavier NX. From your terminal, run the following commands:

sudo sysctl -w net.core.rmem_max=33554432
sudo sysctl -w net.core.wmem_max=33554432
sudo sysctl -w net.core.rmem_default=33554432
sudo sysctl -w net.core.wmem_default=33554432

Relevant links in case anyone wants to do more digging:

1 Like
5

Last link: