This new corpus, called EMO&LY (EMOtion and AnomaLY), is composed of speech and facial video records of subjects that contain controlled anomalies. As far as we know, to study the problem of anomaly detection in discourse by using machine learning classification techniques, no such corpus exists or is available to the community. In EMO&LY, each subject is recorded three times in a recording studio, by filming his/her face and recording his/her voice with a HiFi microphone. During the recording, the subject reads the tale « the handless maiden » from Brothers Grimm displayed as a slideshow.
The corpus is recorded in French. Anomalies in discourse are induced or acted.
The three recordings are structured as follow :
- Introductory record: This is the first time that a participant discovers the slides and reads them in the context of the recording.
- Induced record: For a particular slide, we use an anomaly induction to trigger a reaction.
- Acted record: We ask the participant to act an anomaly for a particular slide. No guideline is given by default for this record. In case the participant asks for directions, we give some basic examples.
At this time, we have recorded 41 subjects for a total of 8,65 hours of usable audiovisual recording, on which we have tested classical classification techniques (GMM or One Class-SVM plus threshold classifier). Our results confirm the usability of the anomaly induction mechanism to produce anomalies in discourse and also the usability of the corpus to improve anomaly detection.
Data available (on request):
- audio features: prosodic features
- video features: landmarks
- information about slide: text content
- information about the anomaly inductor: type of inductor, sample on which it has been triggered and at what time
- manual annotations of the sample (only ones on which an anomaly inductor has been triggered)
- information about each subject : age, gender, french native, self measurement of oral abilities.
To get the data, please write an email to : emoly [at] inria [dot] fr. (Active now ! Thank you for your patience.)
A usage agreement form will be sent back to you with the instructions to get a copy of the corpus.
The raw data could be provided in the future if all the rights provisions are in accordance with the law (image and voice, individual information).
Examples of records:
| Normal: |
|
| Induced: | Acted: |
