djcev.com

There's something wrong with the picture on the left.

Freebsd bktr(4), as configured by libavdevice/bktr.c, reads data coming off the card, fills a memory buffer, then signals libavdevice/bktr.c that a frame is complete. libavdevice is then supposed to copy the frame and process it, while bktr(4) reads new data.

It turns out that libavdevice doesn't wait for that signal before reading a frame. It sleeps for a period of time calculated from the video FPS; a signal from bktr(4) will interrupt this sleep. If no signal is received, meaning the video frame is taking longer to be delivered than expected, then libavdevice will complete the sleep and copy the video buffer anyway.

The above image on the left is the result of copying the video buffer early, without waiting for that frame-complete signal. An image composed of one video frame partially overwritten with another. In this case a fully interlaced frame (above right top) and a non-interlaced frame (above right bottom).

If you've ever tried to record video with ffmpeg's bktr device, this is what that "SLEPT NO signals" message means. It means that libavdevice/bktr.c bktr_getframe() called usleep() for the expected amount of time, didn't receive a signal, and proceeded to read a frame early.

This infuriated me for about a month and a half. After reading the bktr(4) driver and libavdevice/bktr.c about 50 times, I came to the conclusion above. I "fixed" it by altering bktr_getframe() in libavdevice to fully block until bktr(4) sends the frame-complete signal. Which should force a kind of hard frame synchronisation.

My bktr_getframe() looks like this:

static void bktr_getframe(uint64_t per_frame)
{
    uint64_t curtime;
    curtime = av_gettime();

    while (nsignals == 0) {
        /* 834 is 1/40th the duration of an NTSC frame. 
         * More likely I'm an idiot and it's an arbitrary number. */
        usleep(834);
    }

    nsignals = 0;
    last_frame_time = curtime;
}

And that works for me to record interlaced NTSC video without broken frames.

To be clear: this is over my head. I'm not entirely sure my explanation above correctly describes what is going wrong. If I said I fully understood what's happening here I'd be lying.