GeoFidelity-Bench: Evaluating Segment-Level Geographic Fidelity in Text-to-Image Street-View Generation

2026-06-22

AI Summary

Existing text-to-image models struggle to generate urban street scenes consistent with specified road segments, and lack evaluation metrics for segment-level geographic fidelity. This work proposes GeoFidelity-Bench, a novel benchmark comprising 7,117 Mapillary images from 109 OpenStreetMap road segments across 25 cities on six continents, establishing the first evaluation framework focused on segment-level geographic fidelity that prioritizes local discriminability over global similarity. By integrating geospatial data, street-view imagery, and structured textual prompts—including city, street name, neighborhood, and GPS coordinates—and employing reference-panel comparisons and retrieval-based ranking, the study finds that incorporating street and neighborhood names improves Top-1 retrieval accuracy by 5.5 percentage points. Nevertheless, models still fail to reliably distinguish target segments from their nearest same-city counterparts, revealing significant limitations in accurately reconstructing specific road segments.

Abstract

Text-to-image models can generate visually plausible city streets, but whether their outputs correspond to a requested road segment rather than a generic city prior remains unclear. We introduce GeoFidelity-Bench, a reference-panel benchmark for segment-conditioned geographic fidelity in street-view generation. It contains 7,117 curated Mapillary images covering 109 named OpenStreetMap road segments in 25 cities across six continents. For each generated panel, the benchmark ranks the target reference panel against panels from the nearest segment in the same city, other segments in the same city, and segments from other cities, making local discrimination rather than absolute target similarity the primary test. We evaluate six open-weight text-to-image generators under city-only, street-and-neighborhood, and GPS-augmented prompts. Adding street and neighborhood names is associated with an increase of 5.5 percentage points in top-1 retrieval accuracy over city-only prompts, with a 95% confidence interval from 3.4 to 7.7 percentage points. However, the similarity margin between the target and the nearest segment in the same city remains near zero, indicating that local names improve broad local plausibility more than exact segment identity. Prompts that keep the city fixed but use incorrect street or neighborhood names further show that only part of the gain depends on the correct local names, while appending raw GPS coordinates as ordinary text yields no statistically clear additional benefit. Held-out real-image queries successfully recover segment identity, showing that the curated references contain usable segment-level signal. GeoFidelity-Bench thus reveals a persistent gap between city- or neighborhood-plausible street-view generation and faithful generation for a specific road segment.

Problem

• geographic fidelity
• text-to-image generation
• street-view synthesis
• segment-level accuracy
• location conditioning

Innovation

• GeoFidelity-Bench
• segment-level geographic fidelity
• street-view generation
• reference-panel benchmark
• text-to-image evaluation