Modification Design Document: Camera Stream & Isolate Processing

Overview

The goal is to enhance the FingerScanner app to perform real-time biometric quality analysis. We will implement an ImageStream from the camera, send frames to a background Isolate for processing (Sharpness, Luminance, ROI checks), and update the UI with a guide overlay that changes color based on the quality status.

Analysis of the Goal

Real-time Feedback: Users need to know if their finger is placed correctly and if the image quality is sufficient for capture.
Performance: Image processing is CPU-intensive. Doing it on the main thread will cause UI jank. Offloading to an Isolate is necessary.
Throttling: Processing every single frame from the camera (30fps+) is unnecessary and wasteful. We will throttle processing to 5-10fps.
Visual Guide: A CustomPainter is required to draw a dynamic rectangle that responds to the analysis results.

Alternatives Considered

Compute Function: While simpler, compute spawns a new isolate for each call. For a continuous stream, a long-lived Isolate with a message passing mechanism is more efficient.
Main Thread Processing: Rejected due to performance concerns.
Native Code (JNI/FFI) directly: While opencv_dart uses FFI, managing the thread manually in C++ is more complex than using Dart Isolates for this prototype.

Detailed Design

1. Data Layer

CameraDataSourceImpl:
- Add startImageStream(Function(CameraImage) onImage) method.
- Add stopImageStream() method.
- The stream will be managed by the Repository/Provider, which will handle throttling.

2. Domain Layer

QualityStatus Entity:
- double blurScore
- int brightness
- bool isCentered
- bool canCapture (derived from the above)
ScanFingerprint UseCase:
- This might need adjustment, or we introduce a new UseCase MonitorFingerQuality that returns a Stream<QualityStatus>. Given the requirement, we'll likely keep the logic in the Notifier or a dedicated Service to manage the Isolate.

3. Isolate Logic (`ImageProcessor`)

Input: CameraImage (converted to a transferable format if needed, or raw bytes).
Processing:
- Convert: YUV/BGRA -> Grayscale Mat (using opencv_dart).
- Sharpness: cv.Laplacian -> variance.
- Luminance: cv.mean.
- ROI: Crop center -> check pixel density/histogram (simplified segmentation).
Output: QualityStatus object sent back to the main thread.
Throttling: The main thread will control the rate of sending images to the Isolate, or the Isolate can internally throttle. Sending from main thread is easier to control.

4. Presentation Layer

ScannerNotifier:
- Manages the Isolate.
- Subscribes to startImageStream.
- Throttles frames (e.g., using DateTime.now() difference).
- Sends frames to Isolate.
- Updates state with the latest QualityStatus.
ScannerState:
- Add QualityStatus? qualityStatus.
CameraPreviewWidget:
- Replace the static Container overlay with a CustomPaint widget.
- FingerprintGuidePainter: Draws a rectangle with color based on qualityStatus.canCapture (Red vs Green).

5. Dependency Updates

Add opencv_dart to pubspec.yaml.

Diagrams

Architecture Flow

graph TD
    CameraStream[Camera Stream] -->|Raw Frames| Notifier[ScannerNotifier]
    Notifier -- Throttled (5-10fps) --> Isolate[Background Isolate]
    Isolate -- opencv_dart --> Analysis[Analysis: Sharpness, Luminance, ROI]
    Analysis -->|QualityStatus| Notifier
    Notifier -->|Update State| UI[ScannerScreen]
    UI -->|CustomPainter| Guide[Visual Overlay]

4.0 KiB Raw Blame History