Physics-Informed Human Understanding
Unlike traditional pose estimation that relies on statistical pattern matching, AuraSense uses physics and biomechanics to truly understand human movement.
Traditional Approach
- ✕ 2D/3D keypoints from neural networks
- ✕ Black-box predictions with no physical validity
- ✕ Jitter and physically impossible poses
- ✕ Limited to what the model was trained on
AuraSense Approach
- ✓ Physics simulation with mass, inertia, joint constraints
- ✓ Inverse kinematics ensures biomechanical validity
- ✓ Smooth, physically plausible motion
- ✓ Explainable metrics (velocity, joint angles, tremor frequency)
System Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ INPUT LAYER │
├─────────────────────────────────────────────────────────────────────────────┤
│ Video Stream (webcam/depth) │ Audio Stream │ Biosensors (optional) │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ PROCESSING LAYER │
├─────────────────────────────────────────────────────────────────────────────┤
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Initial Pose │ │ Physics Engine │ │ Biomechanics │ │
│ │ Detection │─▶│ (Inverse IK) │─▶│ Validation │ │
│ │ (Neural Net) │ │ │ │ │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Audio Analysis │ │ Facial Sentiment │ │ Multi-Modal │ │
│ │ (MFCC, VAD) │ │ (Valence/Arousal)│ │ Fusion │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ OUTPUT LAYER │
├─────────────────────────────────────────────────────────────────────────────┤
│ Raw Kinematics │ Digital Biomarkers │ Real-time Feedback │ Session Export │
└─────────────────────────────────────────────────────────────────────────────┘
<100ms
End-to-end latency
60fps
Real-time processing
33+
Skeletal joints tracked
Browser
No install required
Built With
🦀
Rust
Core engine, maximum performance
⚡
WebTransport
QUIC-based low-latency streaming
🧠
WebGPU
Browser-native GPU acceleration