The Challenge
Every SLAM paper claims state-of-the-art results — on its own hand-picked benchmark, its own hardware, against its own choice of baselines. That makes it nearly impossible for a team building a real product (a drone, an AGV, an AR headset) to answer a simple engineering question: given our camera, our compute budget, and our accuracy/robustness requirements, which of the dozens of monocular SLAM systems out there should we actually integrate?
We ran that comparison ourselves — 8 named systems spanning classical feature-based tracking, direct photometric odometry, deep recurrent bundle adjustment, and 3D Gaussian-splatting SLAM — evaluated on the same data, the same metrics, and the same brand-new (Blackwell, RTX 5070 Ti) GPU that most of these research repos don't even support out of the box yet.
What We Built
A Build Gauntlet on Unreleased Hardware
Every GPU-accelerated method here pins an old PyTorch/CUDA combination with prebuilt custom kernels — none of them ship support for sm_120 Blackwell silicon. Rather than downgrade the hardware to fit the software, we patched each repo's custom CUDA correlation, rasterization, and bundle-adjustment kernels to compile against a modern PyTorch/CUDA 13.3 stack, built every dependency (Pangolin, DBoW2, UniDepth, custom Gaussian rasterizers) from source, and kept a full build/run log per method for reproducibility.
8/8 methods built and ran successfully:
ORB-SLAM3 classical, indirect (features) — CPU only
DSO classical, direct (photometric) — CPU only
DROID-SLAM deep, dense recurrent flow + BA — GPU
DPVO deep, sparse patch-based VO — GPU
MonoGS 3D Gaussian-splat SLAM — GPU
Splat-SLAM DROID tracking + Gaussian mapping — GPU
HI-SLAM2 DROID tracking + Gaussian + depth — GPU
GigaSLAM metric-depth VO + Gaussian + loop closure — GPUAn Apples-to-Apples Evaluation Harness
All 8 methods were run on the same 5 KITTI raw driving sequences, against a common OXTS GPS/IMU-derived reference trajectory, scored with evo's Sim(3)-aligned Absolute Trajectory Error (the standard way to fairly score scale-ambiguous monocular systems), plus per-run speed and GPU-memory profiling. Two methods that don't ship a point-cloud exporter (ORB-SLAM3, DSO) got a small source addition so every system's internal map could be exported and rendered in an identical style for direct visual comparison.
Results
40 benchmark runs (8 methods × 5 sequences) completed, producing a dataset large enough to separate real signal from noise across accuracy, scale behavior, and robustness:
All 8 methods — spanning four different algorithm families — land within a tight 50-58 m error band on one specific 439-frame sequence with a sharp turn, while scoring sub-2 m on every other route. That's not a bug in any single implementation; it's cross-validated evidence that this particular route genuinely stresses monocular tracking without loop closure — exactly the kind of case a client needs flagged before it becomes a field failure.
DSO — a 2016, CPU-only, classical direct method with zero learned components — has the best or tied-best accuracy on 4 of 5 sequences, ahead of every GPU-accelerated deep-learning method tested. A useful, counter-intuitive data point: "deep learning" isn't a free accuracy upgrade for short, well-conditioned sequences — it buys robustness and dense mapping capability that's worth paying for only when the use case actually needs it.
Only GigaSLAM recovers real-world metric scale directly from a single RGB camera (within 10-40% of ground truth), because it grounds tracking in a metric-depth foundation model rather than scale-free epipolar geometry. Every other method's output units are arbitrary and sequence-dependent — meaning a product built on them needs a separate calibration step (a known object size, GPS, or another sensor) before its trajectories mean anything in the real world. That's a distinction that never shows up in a paper's headline accuracy number, but decides whether a system is deployable as-is.
The accuracy table also hides real robustness differences that matter for shipping a product: MonoGS hung once and crashed once mid-sequence; HI-SLAM2 crashed on its densest keyframe buffer; GigaSLAM hit a recoverable CUDA out-of-memory condition; DSO's own initializer needed up to 28 retries on two sequences (documented upstream behavior, not a benchmark artifact). ORB-SLAM3, DROID-SLAM, DPVO, and Splat-SLAM ran clean across all 40 runs with zero crashes, hangs, or retries.
Technology Stack
Why It Matters
This benchmark is the exact work a robotics, AR/VR, or autonomy client needs before committing engineering months to a SLAM stack: a rigorous, hardware-matched, apples-to-apples comparison — not a literature review. ROVOQ can run this same evaluation against your camera, your compute budget, and your accuracy requirements, and go further: porting the winning system's CUDA kernels onto whatever GPU architecture your product actually ships on, exactly as we did here for hardware most of these research repos don't yet officially support.