77 lines
4.0 KiB
Markdown
77 lines
4.0 KiB
Markdown
# Modification Design Document: Camera Stream & Isolate Processing
|
|
|
|
## Overview
|
|
The goal is to enhance the `FingerScanner` app to perform real-time biometric quality analysis. We will implement an `ImageStream` from the camera, send frames to a background `Isolate` for processing (Sharpness, Luminance, ROI checks), and update the UI with a guide overlay that changes color based on the quality status.
|
|
|
|
## Analysis of the Goal
|
|
* **Real-time Feedback**: Users need to know if their finger is placed correctly and if the image quality is sufficient for capture.
|
|
* **Performance**: Image processing is CPU-intensive. Doing it on the main thread will cause UI jank. Offloading to an `Isolate` is necessary.
|
|
* **Throttling**: Processing every single frame from the camera (30fps+) is unnecessary and wasteful. We will throttle processing to 5-10fps.
|
|
* **Visual Guide**: A `CustomPainter` is required to draw a dynamic rectangle that responds to the analysis results.
|
|
|
|
## Alternatives Considered
|
|
* **Compute Function**: While simpler, `compute` spawns a new isolate for each call. For a continuous stream, a long-lived `Isolate` with a message passing mechanism is more efficient.
|
|
* **Main Thread Processing**: Rejected due to performance concerns.
|
|
* **Native Code (JNI/FFI) directly**: While `opencv_dart` uses FFI, managing the thread manually in C++ is more complex than using Dart Isolates for this prototype.
|
|
|
|
## Detailed Design
|
|
|
|
### 1. Data Layer
|
|
* **`CameraDataSourceImpl`**:
|
|
* Add `startImageStream(Function(CameraImage) onImage)` method.
|
|
* Add `stopImageStream()` method.
|
|
* The stream will be managed by the Repository/Provider, which will handle throttling.
|
|
|
|
### 2. Domain Layer
|
|
* **`QualityStatus` Entity**:
|
|
* `double blurScore`
|
|
* `int brightness`
|
|
* `bool isCentered`
|
|
* `bool canCapture` (derived from the above)
|
|
* **`ScanFingerprint` UseCase**:
|
|
* This might need adjustment, or we introduce a new UseCase `MonitorFingerQuality` that returns a `Stream<QualityStatus>`. Given the requirement, we'll likely keep the logic in the `Notifier` or a dedicated Service to manage the Isolate.
|
|
|
|
### 3. Isolate Logic (`ImageProcessor`)
|
|
* **Input**: `CameraImage` (converted to a transferable format if needed, or raw bytes).
|
|
* **Processing**:
|
|
* **Convert**: YUV/BGRA -> Grayscale `Mat` (using `opencv_dart`).
|
|
* **Sharpness**: `cv.Laplacian` -> variance.
|
|
* **Luminance**: `cv.mean`.
|
|
* **ROI**: Crop center -> check pixel density/histogram (simplified segmentation).
|
|
* **Output**: `QualityStatus` object sent back to the main thread.
|
|
* **Throttling**: The main thread will control the rate of sending images to the Isolate, or the Isolate can internally throttle. Sending from main thread is easier to control.
|
|
|
|
### 4. Presentation Layer
|
|
* **`ScannerNotifier`**:
|
|
* Manages the `Isolate`.
|
|
* Subscribes to `startImageStream`.
|
|
* Throttles frames (e.g., using `DateTime.now()` difference).
|
|
* Sends frames to Isolate.
|
|
* Updates `state` with the latest `QualityStatus`.
|
|
* **`ScannerState`**:
|
|
* Add `QualityStatus? qualityStatus`.
|
|
* **`CameraPreviewWidget`**:
|
|
* Replace the static Container overlay with a `CustomPaint` widget.
|
|
* **`FingerprintGuidePainter`**: Draws a rectangle with color based on `qualityStatus.canCapture` (Red vs Green).
|
|
|
|
### 5. Dependency Updates
|
|
* Add `opencv_dart` to `pubspec.yaml`.
|
|
|
|
## Diagrams
|
|
|
|
### Architecture Flow
|
|
```mermaid
|
|
graph TD
|
|
CameraStream[Camera Stream] -->|Raw Frames| Notifier[ScannerNotifier]
|
|
Notifier -- Throttled (5-10fps) --> Isolate[Background Isolate]
|
|
Isolate -- opencv_dart --> Analysis[Analysis: Sharpness, Luminance, ROI]
|
|
Analysis -->|QualityStatus| Notifier
|
|
Notifier -->|Update State| UI[ScannerScreen]
|
|
UI -->|CustomPainter| Guide[Visual Overlay]
|
|
```
|
|
|
|
## References
|
|
* [Flutter Camera Plugin - ImageStream](https://pub.dev/packages/camera)
|
|
* [Dart Isolates](https://dart.dev/language/concurrency)
|
|
* [OpenCV Dart](https://pub.dev/packages/opencv_dart)
|