More shot validation issues

[cinematicity]

I am having more issues with the "Adjust annotation bounds" feature. It might be a bug in Advene, or maybe I don't understand something, so I'll try to explain.

(1) After I do "detect shots" on my video, I select "Adjust annotation bounds". I notice that every auto-detected annotation begins on the second frame of the shot. This makes me think there is an off-by-one error in the code that processes the results from shotdetect. Would be good if this could be fixed -- it takes a chunk of time to go through 400+ shots in a film to fix this.

(2) Next, if I double-click on an annotation, (for example, let's say it's #278), I get a popup to edit it. The thumbnails for the beginning and ending shots look wrong, because the ending thumbnail is actually from the next shot, #279, even after I adjust it in the shot validation view.

Moreover, the timecode for the end point does not correspond to what I am seeing in the shot validation view. For example, the shot validation view shows 00:59:00.620 as the beginning frame, and 00:59:15.178 as the end frame, but the annotation editing view shows 00:59:00.620 for the beginning and 00:59:15.217 as the end point — not the same. The end point in the edit view is the time code for the first frame of the next shot in the validation view.

At this point, I am confused by what I am seeing. The numbers are not consistent between the various views. Which one is correct? Could there be several off-by-one bugs in the Advene code (i.e. in the shotdetect code, and in the validation view)?

Beyond this, there are a number of rough edges in the GUI that make it slower than necessary to validate annotations.

(3) The "Adjust annotation bounds" popup always starts from annotation #1. Could it start from the currently selected annotation, so I don't have to type in that number every time I need to adjust annotation bounds? (Or, is there some workflow I'm not understanding here?)

(4) By default, the GUI setting for "Open popups" is "as a popup window". The behavior of these windows is really rough. The shot validation popup opens at the edge of the screen with the lower part actually off the screen, and it doesn't remember its location, so I must move the popup window back onto the screen each time.

If I double-click to edit the details of an annotation, and then try to use the previous/next buttons, the whole popup window re-opens and changes to a different size. This behavior is quite awkward, so I finally just switched the popup behavior to "embedded east of the video". IMHO, it might be good to make this the default setting, at least until the problems with popup windows have been fixed.

(5) Finally, this is kind of a minor point but the view menus are labeled "… embedded east/west/south … " -- I have to visualize a map of the world to understand this. Something like "left/right/below" might be more obvious.

Any guidance in understanding this would be appreciated.

[oaubert]

Thanks for all those precise reports, I helps us to improve Advene. Here are some explanations.

(1) After I do "detect shots" on my video, I select "Adjust annotation bounds". I notice that every auto-detected annotation begins on the second frame of the shot. This makes me think there is an off-by-one error in the code that processes the results from shotdetect. Would be good if this could be fixed -- it takes a chunk of time to go through 400+ shots in a film to fix this.

It depends on a number of factors. Advene only manipulates timecodes and does not care about frame rate. Shotdetect on the other side works on frames, and converts their number to a timecode. The trouble is to define what timecode will correctly address a frame, since basically there can be 40ms in a frame (for a 25fps rate).

Now, to pragmatically address your issue:
* check the "Expert mode" in the Edit/Preferences/GUI tab.
* right-click on the annotation type. In the popup submenu, you will find an Offset item. You can enter 40 to offset the bounds of each annotation by 40ms.

(2) Next, if I double-click on an annotation, (for example, let's say it's #278), I get a popup to edit it. The thumbnails for the beginning and ending shots look wrong, because the ending thumbnail is actually from the next shot, #279, even after I adjust it in the shot validation view.

Moreover, the timecode for the end point does not correspond to what I am seeing in the shot validation view. For example, the shot validation view shows 00:59:00.620 as the beginning frame, and 00:59:15.178 as the end frame, but the annotation editing view shows 00:59:00.620 for the beginning and 00:59:15.217 as the end point — not the same. The end point in the edit view is the time code for the first frame of the next shot in the validation view.

At this point, I am confused by what I am seeing. The numbers are not consistent between the various views. Which one is correct? Could there be several off-by-one bugs in the Advene code (i.e. in the shotdetect code, and in the validation view)?

It all comes from the fact that Advene only manipulates timecodes, not frame numbers (and we cannot change this, since we want to be able to annotate audio too, which needs ms precision). The annotation editing window always displays the exact values that are stored in the package.

The issue of the shot validation has the following explanation: this view is based on the begin time for each annotation. It assumes that all annotations are consecutive, with a 1 frame (i.e. 1000/fps ms) difference between the end time of an annotation and the begin time of the following annotation. The shotdetect importer produces consecutive annotations, but the end time of an annotation is equal to the begin time of the following one. This is the cause of the issues in snapshot display, and discrepancies in the shot validation view: the 00:59:15.178 you see for the end frame is I would guess the begin time of the following one (00:59:15.218) minus 40 (1000/25fps).

I though that I fixed the shotdetector to produce correct annotations, but it appears that I did not do it yet. I have just fixed this, and we will try to release a fixed version soon.

Note that fps detection is not easy to do for all files, so for the moment, its value has to be configured in the Edit/Preferences/Time-related dialog.

Beyond this, there are a number of rough edges in the GUI that make it slower than necessary to validate annotations.

(3) The "Adjust annotation bounds" popup always starts from annotation #1. Could it start from the currently selected annotation, so I don't have to type in that number every time I need to adjust annotation bounds? (Or, is there some workflow I'm not understanding here?)

There are 2 ways of doing this:

- you can first click on the annotation you want to begin from in the timeline, then in the shot validation GUI, click on "Current time" to jump to the annotation containing the current player time.

- if you want to adjust a specific annotation, you can move the cursor over the annotation, and use the "a" shortcut to adjust the begin timestamp, and the "A" shortcut to adjust the end timestamp.

(4) By default, the GUI setting for "Open popups" is "as a popup window". The behavior of these windows is really rough. The shot validation popup opens at the edge of the screen with the lower part actually off the screen, and it doesn't remember its location, so I must move the popup window back onto the screen each time.

In fact, there is code to memorize size and position for all popup windows, but depending on the window manager/OS, it is not always possible to restore them.

If I double-click to edit the details of an annotation, and then try to use the previous/next buttons, the whole popup window re-opens and changes to a different size. This behavior is quite awkward, so I finally just switched the popup behavior to "embedded east of the video". IMHO, it might be good to make this the default setting, at least until the problems with popup windows have been fixed.

In fact I use this setting (embed east). I was not quite sure to have definite arguments in favor of one or the other option as default, but I guess you made the point. I will set "east" as default.

(5) Finally, this is kind of a minor point but the view menus are labeled "… embedded east/west/south … " -- I have to visualize a map of the world to understand this. Something like "left/right/below" might be more obvious.

Well... There were some puns intended there, since using east/west would allow use to use far-east/far-west for other possible options.

[cinematicity]

Thanks for the detailed clarification. I am looking forward to the next version of Advene that includes the fix to the shot detector.

[cinematicity]

I though that I fixed the shotdetector to produce correct annotations, but it appears that I did not do it yet. I have just fixed this, and we will try to release a fixed version soon.

I'm belatedly following up on this.

Is there a new version available for OS X that fixes this problem?

[oaubert]

There is no new MacOS X binary release. The evolution of the MacOS environment implies that there is a lot of work to do to do a proper release. However, if you have specific issues, let me know and we can try to find out if it can be worked from the current version (through plugins for instance).

[cinematicity]

Thanks. So does this mean the problem has been fixed for the Windows version of Advene? Also, how can I find out what's been fixed in the latest versions of Advene? I tried looking at the repository, but I can't find the relevant information there. Too much arcane stuff (BTW, I am an SVN user at my job, not a complete noob ).

Also, what would you suggest for diagnosing problems with shotdetect? It has worked for me in the past, but trying again recently I had absolutely no luck with it on a feature-length film. I tried a number of different thresholds, with no effect. Problems: only half of the film was marked before shotdetect quits. Either it misses really obvious shot transitions or else it "finds" transitions where there are none. I tried converting the film with different codecs (mpeg-2, mpeg-4, H.264) but it seems to make no difference. Basically, the output was useless and I had to give up.

[oaubert]

Yes, the windows version should be ok - the 1.0.2 version on windows has been released on 2012/01/26 and the fix for shotdetect import was on 2012/01/20: https://github.com/oaubert/advene/commi ... ade2f8bc48
No idea about the shotdetect shortcomings - they may be due to newer movie encodings that are not well supported in the now aging shotdetect version that is bundled with the Advene installer on macos X and windows. The best way to check would be to use the Linux version, with a recently compiled shotdetect.

[cinematicity]

Unfortunately, I don't have access to a Linux machine. So, it's kind of a show stopper if shotdetect no longer works on OS X or Windows.

If it were possible to compile a new binary of shotdetect on OS X, would that solve the problem? Is there some kind of test suite for shotdetect, to verify that it is working?

[oaubert]

A newly compiled shotdetect might well solve the incomplete shot detections. It must be compiled by statically linking dependencies, so that it can work on other computers than the one where the compilation is taking place.
To test it, you can simply run it from the command line and import the resulting XML file in Advene, to check that everything is OK.

[cinematicity]

Thanks for clarifying. I don't know if it's possible for me to help with this, but I am willing to try some.

I've done a fair amount of code development, but I haven't touched C in twenty years. So, I'm sort of an informed noob, I guess.

Question: which tools do I need to compile shotdetect and do the static linking that you describe? Is it possible to download some tools for free, or are we talking about signing up as a developer somewhere, installing an IDE, paying money, etc.? Next, which libraries do I need to link to, and where do I get them?

Also, about a test suite, I mean some automated way to verify that shotdetect is working, i.e., there are some test files with a few shots and different codecs. The test suite runs shotdetect against them, and then checks the XML output and reports whether it is as expected, or not.

Anything you could tell me to advance this would be very helpful and appreciated. Thanks.