Multiple research themes are covered in the context of the Advene project:
- Interacting with videos
- Multimedia document engineering
- models for video annotation
- enriched video and hypervideo modeling (Cinecast, CHM)
- semantic web and multimedia: see an example and the AdA project,
- Video active reading
- Video active reading: (Cinecast)
- Experience reuse in active reading (Bertrand Richard PhD, Amaury Belin PhD)
The Advene project is or has been involved in the following research projects.
- AdA project (2017-2021) (Audio-Visual Rhetorics of Affect), with the Freie Universität (Berlin) and HPI. Advene is used to thoroughly annotate fiction and documentary movies, in order to carry media studies analyses on filmic representation.
- Museographie (2010-2012) where Advene is used as an analysis tool for reflexive interviews of museum visitors. It contributed to the development of the REMIND method, a methodology for analysing user experience.
- CineCast (2009-2011) that developped collaborative tools for multimodal annotation of audiovisual content, aiming at movie libraries, on-demand video providers and movie-related social networks.
- ACAV, about video accessibility issues.
- Cinelab (2007-2008)
- Incorporactions (2004-2007)
- Daniel Schmitt, Olivier Aubert (2017). REMIND : une méthode pour comprendre la micro-dynamique de l'expérience des visiteurs de musées. in Revue des Interactions Humaines Médiatisées (RIHM) - Journal of Human Mediated Interactions, 2017, Europia, 2017, 17 (2), pp.43-70 (online).
- Olivier Aubert, Joscha Jaeger (2014). Annotating Video with Open Educational Resources in a Flipped Classroom Scenario. OpenCourseWare Conference Global 2014.
- Olivier Aubert, Yannick Prié, Camila Canellas (2014). Leveraging video annotations in video-based e-learning. in 6th International Conference on Computer Supported Education, Barcelona, Spain, April 2014 (article)
- Olivier Aubert (2012) Using Personal Annotations to Enrich Interactions with Videos. 6th Teletask symposium, Potsdam, oct. 2012 (slides, video).
- Madjid Sadallah, Olivier Aubert, Yannick Prié. CHM: an Annotation- and Component-based Hypervideo Model for the Web. in Multimedia Tools and Applications, 2012, 10.1007/s11042-012-1177-y (article).
- Olivier Aubert, Yannick Prié, Daniel Schmitt. Advene as a Tailorable Hypervideo Authoring Tool: a Case Study. in ACM Symposium on Document Engineering, Paris, sep. 2012, 4p. (PDF)
- Madjid Sadallah, Olivier Aubert, Yannick Prié (2011) Component-based Hypervideo Model: high-level operational specification of hypervideos. Short paper in ACM DocEng. Mountain View, California. sep. 2011.
- Madjid Sadallah, Olivier Aubert, Yannick Prié (2011) Hypervideo and Annotations on the Web. Proceedings of Multimedia on the Web Workshop MMWeb2011, sep. 2011 (slides).
- Yannick Prié, Vincent Puig (2008) Nouveaux modes de perception active de films annotés, In Colloque Cinéma, interactivité et société, nov 2008, Poitiers.
- Bertrand Richard and Yannick Prié and Sylvie Calabretto (2008) Towards a unified model for audiovisual active reading, In Tenth IEEE International Symposium on Multimedia. pp 673--678, dec. 2008.
- Olivier Aubert, Yannick Prié (2007) Poster for ICT 2008
- Olivier Aubert and Yannick Prié and Pierre-Antoine Champin (2008) Advene, une plate-forme ouverte pour la construction d'hypervidéos. Ludovia 2008, Ax-les-Thermes, aug. 2008. 10 pp.
- Yannick Prié and Olivier Aubert and Bertrand Richard (2008) Démonstration : Advene, un outil pour la lecture active audiovisuelle. IHM'2008, Metz, sep. 2008. 9 pp.
- Olivier Aubert, Pierre-Antoine Champin, Yannick Prié, Bertrand Richard (2007) Active reading and hypervideo production. in Multimedia Systems Journal, Special Issue on Canonical Processes of Media Production. 6 pp, 2008.
- Olivier Aubert, Yannick Prié (2007) Annotations de documents audiovisuels - Temporalisation et spatialisation. Atelier Interfaces pour l’annotation et la manipulation d’objets temporels : une comparaison des outils et des paradigmes dans le domaine musical et cinématographique, IHM07, Paris, nov. 2007, 2pp.
- Bertrand Richard (2007) Active reading of audiovisual documents, In 15th ACM International Conference on Multimedia, sep. 2007.
- Olivier Aubert, Yannick Prié (2007) Advene: an open-source framework for integrating and visualising audiovisual metadata. Open Source Competition: Technical presentation and overview paper. Proc. of ACM Multimedia Conference 2007, Augsburg, sept. 2007, 4 pp. (PDF)
- Bertrand Richard, Yannick Prié, Sylvie Calabretto (2007) Lecture active de documents audiovisuels : organisation de connaissances personnelles par la structuration d’annotations. in Ingénierie des Connaissances 2007, Grenoble, July 2007, 12 pp.
- Pierre-Antoine Champin, Yannick Prié (2007) Models for sustaining emergence of practices for hypervideo. in [International Workshop On Semantically Aware Document Processing And Indexing SADPI'07, May 2007, 10pp.
- Pierre-Antoine Champin, Yannick Prié (2007) On the Articulation of Document and Ontological Models: Integration of Semantic Web Technology in an Annotation-based Hypervideo System. 28 pp. LIRIS Research Report
- Olivier Aubert, Pierre-Antoine Champin, and Yannick Prié (2006). Integration of semantic web technology in an annotation-based hypervideo system. SWAMM 2006, First International Workshop on Semantic Web Annotations for Multimedia, 2006. 12 pp.
- Olivier Aubert and Yannick Prié (2006) Des vidéos aux hypervidéos : vers une autre interaction avec les médias audiovisuels, in TSI - numéro spécial Document numérique Vol 25/4, 2006 (in french).
- Olivier Aubert and Yannick Prié (2005) Des hypervidéos pour créer et échanger des analyses de documents audiovisuels, H2PTM'05, Nov-Dec 2005 (in french). (article, slides)
- Olivier Aubert and Yannick Prié (2005) Advene: active reading through hypervideo., ACM Hypertext'05, Sep 2005. (article, slides)
- Olivier Aubert and Yannick Prié (2004) Documents audiovisuels instrumentés - Temporalités et détemporalisations dans les hypervidéos, in Document numérique, numéro spécial "Temps et Document", Vol. 8/4, Dec 2004, pp. 143-168 (in french) (article)
- Olivier Aubert and Yannick Prié (2004) From video information retrieval to hypervideo management, Proceedings of Corimedia, the international workshop on multidisciplinary image, video, and audio retrieval and mining, Sherbrooke, Quebec, Oct. 2004 (article)
- Olivier Aubert, Pierre-Antoine Champin and Yannick Prié (2004) Advene : une plate-forme pour l'édition et la visualisation d'hypervidéos, , in Journée Modèles Documentaires de l'Audiovisuel (Semaine du Document Numérique 2004*), La Rochelle, Jun 2004 (article) (in french)
- Olivier Aubert, Pierre-Antoine Champin and Yannick Prié (2004) The Advene Model for Hypervideo Document Engineering, LIRIS research Report, RR-2004-22, University Claude Bernard Lyon1, Jan 2004 (article)
- Olivier Aubert, Pierre-Antoine Champin and Yannick Prié (2003) Instrumentation de documents audiovisuels : temporalisations et détemporalisation dans le projet Advene, in Workshop "Temps et documents numériques", Grenoble, Nov. 2003 (in french)
Advene has been and is being used in various projects. You can find here a non-comprehensive list of uses of Advene.
- The CineCast FUI project aims at studying and deploying movie enrichment and audiovisual metadata exchange.
- Daniel Schmitt from LISEC (Laboratoire interuniversitaire des Sciences de l'Education et de la Communication) Faculté des Sciences de l'Education, Université de Strasbourg -- has used Advene during his PhD to analyse video interviews of museum visitors. Its results can be seen on the museographie.fr website. It is a nice example of how Advene covers multiple aspects of video annotation: edition, video exploration and visualisation through user-defined templates.
- The Technology and Design for Healthcare research group (from Politecnico de Milano) uses Advene to analyse video and augment them with sensor information, such as in the Babylandia project.
- We collaborated with Livia Giunti (Pisa University), who used Advene to analyse fiction movies in the context of her PhD in History of Art Problems of fiction film analysis: testing and development of an analytical and interpretive model with digital tools (Problemi dell'analisi del testo di finzione audiovisivo: verifica e sviluppo di un modello analitico e interpretativo con strumenti digitali). See also Delle gioie e dei dolori dell’analisi del film (Italian) and L’analyse du film a l’ère numérique. Annotation, geste analytique et lecture active (French).
- The TIMC-IMAG laboratory uses Advene to annotate video recordings in the context of the Health Smart Home project: A Multimodal Corpus Recorded in a Health Smart Home.
- Maxim Makatchev, Reid Simmons and Majd Sakr (from Carnegie Mellon University) used Advene to analyse nonverbal behaviors in receptionist interactions.
- M. Rohrbach, S. Amin, M. Andriluka and B. Schiele used Advene to build a fine-grained activity database around cooking: A Database for Fine Grained Activity Detection of Cooking Activities
- The Cine Lab ANR Project aims at studying, developing and experimentally validating a software to annotate movies and produce new contents from these annotations.
- The TeXMeX team at IRISA uses Advene to visualise and validate the output of their video analysis algorithms. They contributed alignment features as well as a new shot detection algorithm.
- In the Actors & Avatars project at Roskilde Universitet (RUC), Advene is used to analyse videoviews, i.e. video interviews and captures of users of virtual worlds, and carry out their investigation.\ Jane Widtfeldt Meged has used Advene for his PhD The Guided Tour - a Co-produced Tourism Performance, to analyse video captures of guided tours and interviews of participants.
- The Synote project used Advene for initial prototypes and used it as inspiration for user interface design and annotation making from Advene.
- INcorporACTIONS - Réincorporer le langage en interaction
Use case scenarios
Some feature movies that have been studied so far, be it for film critics or teaching:
- Mon Oncle
- Mulholland Drive
- AdA project from FUB.
Advene is also used in the sociology/anthropology domain, to annotate and investigate video interviews or video captures of experiments. See the museographie.fr website for example and data files.
This use scenario of our framework is based upon real experiments (screenshot), upon which we will explain various points. Miss X, a foreign language teacher wants to use a movie as a pedagogical exercise. Her class consists in two steps:
- Have the pupils watch the movie by sequences. While the movie plays, watching tips are displayed on the screen (important vocabulary, grammar points, ...). At the end of each sequence, the video player automatically stops.
- Give an exam about the movie. The questions are illustrated by some screen captures of the movie, and the important vocabulary is given again on the exam sheet.
To prepare this class, Miss X uses the Advene prototype. She has to define a structure of annotations relevant for her pedagogical activity, annotate the movie according to this structure and eventually use the result with her pupils.
An educational institution provides on its web server annotation schemas designed for this kind of activity. Each schema defines various annotation types and relation types, related to the activity (denoting a specific dimension of analysis). For example, the watching-tips schema defines the following annotation types: character (annotating a meaningful occurrence of a character), vocabulary (annotating the occurrence of a difficult word, with its definition), caption (holding subtitles for the studied movie).
Generic views are also bundled with the schemas, displaying some of the annotations in a meaningful way. For instance, a User-Time Based View vocabulary-list produces a hypertext document listing all the difficult words of the movie, and allows to visualise the moment of occurrence of each word. A Stream-Time Based View with-watching-tips produces an augmented playback of the stream with subtitles and visual cues added (from the watching-tips schema) to the original movie.
Mr Y, one of Miss X's colleagues, already studied the movie in his class and thus created vocabulary annotations and defined video sequences, using the same schemas. He gives her the XML file holding his annotations, for her to reuse. They do not have to exchange DVDs, both of them having their own copy.
Using the same schemas as Mr Y, Miss X can reuse, by copying or referencing, some of his annotations. She can add her own annotations (for instance, vocabulary more relevant to the level of her class). She can even design and use her own schema, corresponding to a kind of exercise not covered by the institution's schemas. Finally, she writes another document as a UTBV, illustrated with movies screenshots and annotation contents, that she will use as an exercise sheet for her pupils.
At the beginning of the class, the teacher distributes a sheet with the difficult vocabulary list, generated by the generic UTBV vocabulary-list from the institution's schemas, so that the pupils can have a written support.
Then the pupils watch the STBV with-watching-tips which displays the movie enriched with the watching-tips annotations and stops at the end of each sequence. Finally, the teacher distributes the exercise sheet that she wrote, enriched with annotation data and movie screenshots.