# Modification Design Document: Camera Stream & Isolate Processing ## Overview The goal is to enhance the `FingerScanner` app to perform real-time biometric quality analysis. We will implement an `ImageStream` from the camera, send frames to a background `Isolate` for processing (Sharpness, Luminance, ROI checks), and update the UI with a guide overlay that changes color based on the quality status. ## Analysis of the Goal * **Real-time Feedback**: Users need to know if their finger is placed correctly and if the image quality is sufficient for capture. * **Performance**: Image processing is CPU-intensive. Doing it on the main thread will cause UI jank. Offloading to an `Isolate` is necessary. * **Throttling**: Processing every single frame from the camera (30fps+) is unnecessary and wasteful. We will throttle processing to 5-10fps. * **Visual Guide**: A `CustomPainter` is required to draw a dynamic rectangle that responds to the analysis results. ## Alternatives Considered * **Compute Function**: While simpler, `compute` spawns a new isolate for each call. For a continuous stream, a long-lived `Isolate` with a message passing mechanism is more efficient. * **Main Thread Processing**: Rejected due to performance concerns. * **Native Code (JNI/FFI) directly**: While `opencv_dart` uses FFI, managing the thread manually in C++ is more complex than using Dart Isolates for this prototype. ## Detailed Design ### 1. Data Layer * **`CameraDataSourceImpl`**: * Add `startImageStream(Function(CameraImage) onImage)` method. * Add `stopImageStream()` method. * The stream will be managed by the Repository/Provider, which will handle throttling. ### 2. Domain Layer * **`QualityStatus` Entity**: * `double blurScore` * `int brightness` * `bool isCentered` * `bool canCapture` (derived from the above) * **`ScanFingerprint` UseCase**: * This might need adjustment, or we introduce a new UseCase `MonitorFingerQuality` that returns a `Stream`. Given the requirement, we'll likely keep the logic in the `Notifier` or a dedicated Service to manage the Isolate. ### 3. Isolate Logic (`ImageProcessor`) * **Input**: `CameraImage` (converted to a transferable format if needed, or raw bytes). * **Processing**: * **Convert**: YUV/BGRA -> Grayscale `Mat` (using `opencv_dart`). * **Sharpness**: `cv.Laplacian` -> variance. * **Luminance**: `cv.mean`. * **ROI**: Crop center -> check pixel density/histogram (simplified segmentation). * **Output**: `QualityStatus` object sent back to the main thread. * **Throttling**: The main thread will control the rate of sending images to the Isolate, or the Isolate can internally throttle. Sending from main thread is easier to control. ### 4. Presentation Layer * **`ScannerNotifier`**: * Manages the `Isolate`. * Subscribes to `startImageStream`. * Throttles frames (e.g., using `DateTime.now()` difference). * Sends frames to Isolate. * Updates `state` with the latest `QualityStatus`. * **`ScannerState`**: * Add `QualityStatus? qualityStatus`. * **`CameraPreviewWidget`**: * Replace the static Container overlay with a `CustomPaint` widget. * **`FingerprintGuidePainter`**: Draws a rectangle with color based on `qualityStatus.canCapture` (Red vs Green). ### 5. Dependency Updates * Add `opencv_dart` to `pubspec.yaml`. ## Diagrams ### Architecture Flow ```mermaid graph TD CameraStream[Camera Stream] -->|Raw Frames| Notifier[ScannerNotifier] Notifier -- Throttled (5-10fps) --> Isolate[Background Isolate] Isolate -- opencv_dart --> Analysis[Analysis: Sharpness, Luminance, ROI] Analysis -->|QualityStatus| Notifier Notifier -->|Update State| UI[ScannerScreen] UI -->|CustomPainter| Guide[Visual Overlay] ``` ## References * [Flutter Camera Plugin - ImageStream](https://pub.dev/packages/camera) * [Dart Isolates](https://dart.dev/language/concurrency) * [OpenCV Dart](https://pub.dev/packages/opencv_dart)