VisionMate

Background

VisionMate was our S8 main project at APJ Abdul Kalam Technological University. We were a team of four and this was the big one -- the final year project that's supposed to represent what you're capable of before you graduate.

Picking the right problem mattered. We wanted something that was technically ambitious but also genuinely useful, not just a demonstration of a concept that nobody would actually use. Accessibility tech for visually impaired users felt like the right call. The problem is real, the technology to address it exists, and combining it into something practical is the hard part.

What the app does

VisionMate turns an Android phone into an audio-first assistive companion. The entire interface is designed to be used without looking at the screen -- no precise touch required, no visual feedback needed. A visually impaired user can operate it entirely through tap gestures, hardware buttons, and voice commands.

Obstacle detection: The camera continuously scans the environment and announces what's in front of the user. "Car on your left." "Person directly ahead." The system uses ML Kit to classify objects across 400+ categories and estimates proximity based on confidence scores.

OCR text reading: Point the camera at a sign, menu, label, or any printed text and the app reads it aloud. Recognition runs on the ML Kit text recognition engine.

GPS navigation: The user says "I need to go to Vytila" and the app calculates a walking route and gives turn-by-turn audio directions. It detects when the user goes off-route and recalculates automatically.

SOS alert: A long press of the volume-up button (holding for 800ms) sends an SMS to a pre-configured emergency contact with the user's name, current GPS coordinates, and any medical conditions they've entered in their profile. This works even from a locked screen.

Control model

The controls were designed carefully because they determine whether someone can actually use this independently:

Every interaction has audio confirmation. The app tells you what mode you're in and what it's doing.

The background engine

The most important architectural decision was keeping the core functionality alive even when the app is in the background or the screen is off. This is what makes the app actually safe to use -- obstacle announcements should keep coming even if the phone is in a jacket pocket.

This is handled by a foreground service (VisionEngineService) that runs persistently. It owns the camera stream, the ML inference pipelines, the text-to-speech queue, and the wake lock that prevents Android from sleeping the process. The Jetpack Compose UI connects to this service and observes its state -- but the actual work happens in the service, independent of whether the UI is visible.

Alert throttling

A naive obstacle detection system would be unusable -- it would announce the same car every frame for ten minutes. The alert throttle manager applies per-label cooldowns:

There's one override: anything detected as "very near" bypasses all cooldowns and announces immediately. Safety takes priority over a clean audio experience.

GPS navigation detail

The navigation screen was the most complex piece in the project. It uses Google Directions API as the primary source for walking routes and Mapbox Directions as a fallback. Route deviation is detected at a 100-meter threshold and triggers automatic recalculation. Upcoming turns are announced at 100 meters ("prepare to turn") and again at 30 meters ("turn now"). During navigation, obstacle detection keeps running in the background so the user isn't flying blind while following directions.

My contribution

I was responsible for the overall application architecture, the foreground engine service that keeps everything running, the accessibility service for hardware key interception, the home screen and onboarding flow, and the settings system. We used AI-assisted development throughout to move faster on implementation once the design decisions were made -- for a project of this scope, iterating quickly was essential.

Tech stack