Problem 4
Watch how the answer is delivered, push back, and score both the argument and the presence.
Whether preparing for a technical job interview, a legal deposition, or a competitive debate, practicing against a mirror is ineffective. You need an opponent who can watch and listen to your argument, read how you present it, find the flaws, and push back intelligently.
Static lists of interview questions or debate prompts are predictable, and audio-only practice tools miss half the picture. In real high-pressure exchanges — job interviews, depositions, competitive debates — how you say something and how you carry yourself (eye contact, posture, facial expression) matter as much as what you say. True preparation requires an adversary that can see and hear you, adapt to your specific arguments in real-time, ask follow-up questions, and evaluate your performance objectively — verbally and non-verbally.
This problem is relevant to organizations and individuals preparing for high-pressure verbal exchanges, including:
The solution should have the potential to become a reusable product or platform capable of adversarial practice across different domains.
Build an AI system designed to test a user's knowledge and presence by actively arguing against them on video, simulating a high-pressure interview, or throwing unexpected curveballs, followed by a detailed performance review.
The system should engage the user in a dynamic video exchange, read both what the user says and how they present it, adapt its line of questioning based on the user's answers, and produce an objective evaluation of their performance.
Participants are not required to support every format. Teams may choose the formats and difficulty levels they want to address and are encouraged to innovate beyond the examples provided.
Adversarial Reasoning
Ability to spot logical fallacies or technical inaccuracies in user input.
Dynamic Branching
Change the direction of the conversation based on the user's previous answer.
Multimodal Perception
Analyze the user's live video feed — facial expression, eye contact, and posture — alongside their spoken or typed answers to gauge confidence and engagement.
Objective Evaluation
Grade the user fairly against a hidden rubric that combines content quality with communication presence.
The following levels are intended as guidance for increasing complexity. Participants may choose one or more levels based on their approach.
Level 1 (Basic Agency)
The agent asks 5 predefined interview questions over text or video. It collects the user's answers and outputs a final grade with basic feedback.
Level 2 (Tool Integration)
The agent dynamically adjusts difficulty based on answer quality, and samples the user's webcam feed at intervals to detect basic engagement signals (e.g., looking at the camera vs. looking away, or a face being present at all). If the user easily answers a coding question, the agent searches a database for a harder, related follow-up question; if they fail, it pivots to an easier fundamental concept; if engagement visibly drops, it adjusts its tone or re-engages the user.
Level 3 (Multi-Agent Collaboration)
The user interacts with the "Interviewer Agent" over video. A "Fact-Checker Agent" secretly verifies the user's claims in the background (e.g., "I scaled a database to 10 million TPS on a single node"), while a "Body-Language Agent" watches the video stream for stress or confidence cues (e.g., broken eye contact, fidgeting) at the moment a claim is made. Together they prompt the Interviewer to call out impossible, incorrect, or unconvincingly delivered claims.
Level 4 (Advanced / Video-Native, Multimodal)
A fully video-based interaction: the agent processes the live webcam stream (vision) and microphone (Speech-to-Text) together, analyzing facial expression, eye-contact percentage, posture, and vocal confidence markers (filler words, pace, tone) in sync with the content of each answer. It produces a combined "Communication Confidence" score that reflects both verbal and non-verbal delivery, and can reference specific moments on the video timeline (e.g., "At 02:14, when asked about the outage, eye contact dropped and you paused for 4 seconds").
A dynamic, challenging video back-and-forth session ending with a comprehensive evaluation report.
A generated evaluation report may include:
Non-verbal / body-language notes (eye contact, posture, expression analysis) with timestamps
Combined communication confidence score (verbal + visual)
Participants are free to decide:
Which video/vision models or tools (real, mocked, or pre-recorded clips) are used for non-verbal analysis How video input is captured (live webcam, uploaded video, or sampled frames) The system architecture Whether multiple AI models or agents are used Teams may use real, synthetic, or publicly available question banks and sample interview videos where appropriate.
1. Meaningful AI Usage
The solution should demonstrate why intelligent, adaptive, multimodal questioning is required.
Solutions should go beyond a static, predefined question list or audio-only analysis.
2. Questioning and Evaluation Should Be Connected
The user's verbal answers and visual delivery should meaningfully influence the next question and the final assessment.
3. Fairness Preservation
The solution should attempt to keep grading objective and consistent against the stated rubric, and should not penalize incidental video issues (e.g., poor lighting, webcam quality) as if they were confidence signals.
4. Appropriate Processing for Different Formats
Teams supporting multiple formats (technical, behavioral, debate, sales) should demonstrate how their approach adapts to differences in evaluation criteria.
5. Privacy Awareness
The solution should handle video and biometric-adjacent signals (facial expression, eye contact) responsibly, process them locally or transiently where possible, and avoid storing raw video beyond what the demo requires.
6. Actionable / Usable Output
The performance report should be structured so that it can be meaningfully used for improvement.
7. Efficiency
Participants should consider video frame-sampling rate, LLM and vision-model consumption, processing time, cost, and scalability. Analyzing every video frame without considering efficiency should not automatically be considered a better solution.
8. Innovation Freedom
Participants are not restricted to a specific model, framework, scoring approach, architecture, or technology.
What It Measures
How intelligently the agent challenges and follows up
Fairness and correctness of the final grade
Non-Verbal Analysis
Quality
Accuracy and usefulness of body-language and confidence signals derived from video
Ability to branch based on the user's verbal and visual responses
Whether AI provides meaningful value beyond a static script or audio-only analysis
Architecture, implementation, and robustness
Effective use of models, video processing, computation, cost, and latency
Novelty of the adversarial, evaluation, or multimodal approach
Potential to support many concurrent video sessions or formats
Ability to evolve into a practical practice product
Ability to clearly demonstrate the working solution on video
Participants are free to choose their own technical approach.
Solutions may use any relevant combination of:
Computer vision / video models for facial expression, eye-contact, and posture analysis
Speech-to-Text and Text-to-Speech
Multimodal (video + audio + text) models
Fact-checking / retrieval tools
Rubric-based scoring models
Multi-agent orchestration
Agentic AI
The listed technologies are examples only and should not limit innovation.
Given a user's live video responses, intelligently probe, challenge, and adapt like a real adversary — reading both what they say and how they present it — then deliver an objective, rubric-based performance evaluation.