Boosting 3D Foundation Models with
Edge-based Pose Optimization

Mattia D'Urso1 Christian Sormann2 Mattia Rossi2 Friedrich Fraundorfer1
1Graz University of Technology 2Sony Europe
ECCV 2026
Paper Poster Video Code TerraSky3D

News

  • Sept 2026 — EPO will be presented at the 3rd NeuSLAM [Nectar Track], ECCV 2026 Workshop, Sept 8, Malmö.
  • Aug 2026 — EPO v1.2 is out 🚀 2.4× faster than v1.1, same accuracy. Refining 150 cameras takes less than 7 s on a RTX 4090.
  • Jul 2026 — EPO v1.1 adds support for depth (re-)densification with Any2Full.
EPO refining a VGGT reconstruction of the Graz Town Hall scene (TerraSky3D)
EPO refining a VGGT reconstruction of Graz Town Hall (TerraSky3D). Starting from the raw feed-forward output, our trackless optimization aligns image edges to recover accurate camera poses and geometry with no feature extraction or track construction. The ground truth is shown for reference.

TL;DR

EPO runs on top of any 3D Foundation Model, skips feature extraction and matching entirely, and significantly improves the estimated camera poses in a fraction of the model’s own inference time. With densification, it gives back a dense output just like the one the 3DFM would produce — but with better geometry, thanks to the optimization. In short: faster, better, and denser than traditional Bundle Adjustment.

Contributions

  • We propose a robust and trackless optimization framework that boosts the geometric accuracy of 3D Foundation Models through neural pose refinement and first-order optimization.
  • We introduce an edge-based reprojection loss and a pose-based early-stopping criterion that together reduce runtime to a few seconds, compared to the minutes needed by Bundle Adjustment-based refinement methods, while requiring lower memory.
  • We provide a comprehensive evaluation across diverse 3D Foundation Models, benchmarks, and real-world scenarios — including TerraSky3D, ScanNet++, and Mip-NeRF 360 — demonstrating that EPO consistently performs on par with, or superior to, state-of-the-art refinement methods.

How It Works

Given the initial geometry from a 3D Foundation Model, EPO computes for each image a Canny edge map and its Distance Transform Field (DTF), where every pixel encodes the distance to the nearest edge. Rather than matching features, EPO minimizes a bidirectional edge reprojection loss: edges from one view are reprojected into another and scored against the DTF. We learn, per scene, a pose MLP, a focal scaling, and a pixel-wise affine depth correction, all updated by a first-order optimizer until a pose-based stopping criterion is met.

Input RGB image Raw VGGT depth Distance Transform Field
Input RGB image, raw VGGT depth, and the corresponding Distance Transform Field (DTF) from Graz Townhall (TerraSky3D).

Before

After

Edge alignment with the initial geometry, view 1 Edge alignment with the initial geometry, view 2
Edge alignment after EPO, view 1 Edge alignment after EPO, view 2
Bidirectional edge alignment using the initial geometry (left) and after refining camera parameters and depth with EPO (right).

Pose Accuracy & Runtime

Using VGGT as a representative 3D Foundation Model, we compare against VGGT+BA and VGGT+Ref+BA which tracks are predicted by a CoTracker module. +Ref indicates further track refinement step. EPO improves pose accuracy (AUC@5°) on every benchmark while drastically cutting end-to-end runtime — and, unlike +Ref+BA (which needs ≥40 GB of VRAM), it runs on a consumer-grade RTX 4090.

AUC@5° ↑

Runtime (s) ↓

VGGT VGGT+BA VGGT+Ref+BA† VGGT+EPO (torch) VGGT+EPO (triton)

AUC@5° (higher is better) and total end-to-end wall-clock time in seconds (lower is better), averaged per dataset. EPO (torch) and EPO (triton) return the same AUC up to numerical noise, so the AUC bar is shown split between the two. † run on an NVIDIA H200 due to its higher memory requirements; all other methods run on an RTX 4090. These are the numbers reported in the paper, as of June 30, 2026.

Generalization Across 3D Foundation Models

Beyond RGB images for edge extraction, EPO needs only what any 3D Foundation Model already outputs — camera intrinsics, poses, and depth maps — so it drops in on top of different backbones unchanged. Applied to VGGT, MapAnything, π3X, VGGT-Ω, Deja View, and DA3-Large, EPO consistently boosts pose accuracy across every dataset.

VGGT

MapAnything

π³X

VGGT-Omega

Deja View

DA3-L

AUC@5° (higher is better) for six 3D Foundation Models, shown as raw output and after EPO refinement. EPO improves geometric accuracy across all models and datasets. These are the numbers reported in the paper, as of June 30, 2026.

Novel View Synthesis

As a downstream proxy for reconstruction quality, we train a 3DGS model for 30,000 steps on each SfM reconstruction. On Mip-NeRF 360, VGGT+EPO yields the best PSNR, SSIM, and LPIPS among the optimization methods, preserving sharper structural detail than BA-based refinement.

VGGT+Ref+BA vs GT

VGGT+EPO (Ours) vs GT

Garden

Garden — VGGT+Ref+BA Garden — GT
⇆
GTRef+BA
Garden — VGGT+EPO Garden — GT
⇆
GTEPO

Flowers

Flowers — VGGT+Ref+BA Flowers — GT
⇆
GTRef+BA
Flowers — VGGT+EPO Flowers — GT
⇆
GTEPO

Kitchen

Kitchen — VGGT+Ref+BA Kitchen — GT
⇆
GTRef+BA
Kitchen — VGGT+EPO Kitchen — GT
⇆
GTEPO
Qualitative Novel View Synthesis on Mip-NeRF 360 (Garden, Bicycle, Flowers, Kitchen). Drag each slider to reveal the ground-truth held-out view (left) against VGGT+Ref+BA or VGGT+EPO (right).
Method PSNR↑ SSIM↑ LPIPS↓
GT (reference)26.690.8310.209
VGGT20.560.5550.422
VGGT+BA22.700.6610.339
VGGT+Ref+BA23.060.6880.320
VGGT+EPO (Ours)23.930.7210.284

Mean Novel View Synthesis quality on Mip-NeRF 360 (3DGS, 30k steps).

BibTeX

@inproceedings{durso2026epo,
        title={Boosting 3D Foundation Models with Edge-based Pose Optimization},
        author={D'Urso, Mattia and Sormann, Christian and Rossi, Mattia and Fraundorfer, Friedrich},
        booktitle={European Conference on Computer Vision (ECCV)},
        year={2026}
      }