PrepArt
Final year project: a real-time multimodal mock-interview system that fuses facial and eye cues, vocal prosody, and transcript analysis into structured interview feedback.
- Python
- MediaPipe
- Librosa
- MoviePy
- PyTorch
- FastAPI
PrepArt was my final year project at UET Taxila, advised by Dr. Qamas Gul. It is a real-time mock-interview system that records a practice interview and returns feedback grounded in three modalities rather than one.
Why multimodal
Interview performance is not carried by the transcript alone. What a candidate says, how they say it, and how they present while saying it are separable signals, and a system that reads only one of them gives advice that is confidently incomplete. PrepArt extracts each independently and fuses them at the feedback stage.
The three streams
Visual (MediaPipe). Face mesh and iris landmarks are tracked per frame to derive eye-contact proportion, gaze stability, head pose, and coarse expression dynamics over the answer window.
Acoustic (Librosa). Vocal prosody features: pitch contour and variance, energy, speaking rate, and the distribution of pauses. Filler-heavy, monotone, or rushed delivery shows up here even when the transcript reads well.
Textual (transcript analysis). The answer is transcribed and analysed for structure, relevance to the question, specificity, and filler density. The question here is whether an answer actually answers what was asked.
MoviePy handles the media pipeline by segmenting recordings by question, extracting audio tracks for the acoustic stage, and assembling annotated playback so a candidate can see the moment a piece of feedback refers to.
Feedback design
The output is deliberately not a single score. A number invites optimisation of the number. Instead each modality produces per-question observations with the evidence attached (the timestamp, the measured value, and the comparison), so the feedback is checkable by the person receiving it.