Rice University School of Engineering and Computing, Department of Electrical and Computer Engineering

Visual Computing Systems · MobiSys 2021

LensCap: Split-Process Framework for Fine-Grained Visual Privacy Control for Augmented Reality Apps

Authors
Jinhan Hu, Andrei Iosifescu, Robert LiKamWa
Venue
MobiSys 2021 - The 19th International Conference on Mobile Systems, Applications, and Services
ACM Digital LibraryBibTeX

LensCap restructures AR applications so visual processing and network access are separated, giving people meaningful control over what visual data can leave their device.

Introduction: Visual Privacy in AR

AR applications continuously capture a user’s physical environment. When an app also has network access, camera frames and derived visual information can be collected without meaningful user awareness or control.

System Architecture: Separating Vision From Connectivity

LensCap divides an AR application into a visual process that can operate on camera frames but cannot access the network, and a network process that can communicate externally but cannot access the camera. It monitors the transactions between them and presents those choices to the user at a fine granularity.

LensCap split-process privacy workflow
Figure 1. LensCap mediates the data shared between visual and network processes.
LensCap split-process access control design
Figure 2. Split-process access control separates camera access from network access.

Results: Control Without Disrupting AR

The framework supports expressive on-device visual processing while allowing people to decide which forms of data are sent to the network. Its Android-library prototype and Unreal Engine plugin preserve visual privacy with an insignificant latency penalty in evaluated AR applications.

LensCap Unreal Engine blueprint example
Figure 3. LensCap exposes visual-data access through an Unreal Engine workflow.
LensCap performance evaluation
Figure 4. Evaluated applications retain their performance with LensCap’s access control.
Comparison of LensCap data use notifications
Figure 5. LensCap gives people contextual visibility into when visual data is shared.