Google opens SafetyCore content-scanning API to third-party Android apps

What SafetyCore does

Android users were surprised last year to find Android System SafetyCore installed on their devices without consent. The app performs image classification on the device to prevent users from seeing nudity when they receive, send, or forward messages in Google Messages. Google states that classification runs exclusively on the device and results are not shared with Google.

SafetyCore has no icon and no clear explanation of what it does, which fueled distrust. A member of the r/degoogle community reported seeing the app reinstall itself, with a screenshot showing SafetyCore reappearing on their device. Some users sideload a placeholder app with an inflated version number, SafetyCore v2000000000, to stop Google Play Store from overwriting the real app.

A new public API

Google's developer documentation now describes a public API in development called ContentSafetyManager. It allows other Android apps to request the same kind of on-device content scan. The API is currently added in API level 10000, a special placeholder code representing the current development version.

The API supports images as well as other media types, and returns four broad categories: allowed, warning, blocked, and unclassified. Apps can then decide whether to display or blur the content. The documentation does not mention whether the API uses SafetyCore specifically, and there is no nudity category in the classification scheme.

Privacy concerns

Cybernews information security researcher Rasa Jurgutyte said she was not surprised by Google's move but worried about public awareness. She said that even someone fully immersed in tech can miss such changes, and that people are normalizing a level of surveillance they do not fully understand.

Experts warn that a malicious app with excessive permissions could abuse the content-scanning capability to inspect large amounts of user content, enabling privacy-invasive profiling or worse. However, content classification alone does not grant an app access to all photos and files on the device.

SafetyCore requires five permissions: view network connections, full network access, run at startup, prevent device from sleeping, and read Google service configuration. According to the Google Play Store Data Safety tab, no data is collected or shared with third parties. The app has over 1 billion downloads and a 3.6 star average from nearly 240,000 reviews.

Apple uses a similar on-device system called Communication Safety, and its SensitiveContentAnalysis framework lets other apps tap into the same detection feature.

Sources: Cybernews