Buffer underrun causes timeout

If I don’t call arv_stream_timeout_pop_buffer() often enough and Aravis runs out of input buffers and gets buffer underruns, after a while (a few calls) I start to get ARV_BUFFER_STATUS_TIMEOUT status on my buffers. Why does buffer underrun cause timeouts to happen? What can I do about it?

Gigevision-ethernet protocol.