Skip to content

Commit

Permalink
feat(mobile): native_video_player (#12104)
Browse files Browse the repository at this point in the history
* add native player library

* splitup the player

* stateful widget

* refactor: native_video_player

* fix: handle buffering

* turn on volume when video plays

* fix: aspect ratio

* fix: handle remote asset orientation

* refinements and fixes

fix orientation for remote assets

wip separate widget

separate video loader widget

fixed memory leak

optimized seeking, cleanup

debug context pop

use global key

back to one widget

fixed rebuild

wait for swipe animation to finish

smooth hero animation for remote videos

faster scroll animation

* clean up logging

* refactor aspect ratio calculation

* removed unnecessary import

* transitive dependencies

* fixed referencing uninitialized orientation

* use correct ref to build android

* higher res placeholder for local videos

* slightly lower delay

* await things

* fix controls when swiping between image and video

* linting

* extra smooth seeking, add comments

* chore: generate router page

* use current asset provider and loadAsset

* fix stack handling

* improved motion photo handling

* use visibility for motion videos

* error handling for async calls

* fix duplicate key error

* maybe fix duplicate key error

* increase delay for hero animation

* faster initialization for remote videos

* ensure dimensions for memory cards

* make aspect ratio logic reusable, optimizations

* refactor: move exif search from aspect ratio to orientation

* local orientation on ios is unreliable; prefer remote

* fix no audio in silent mode on ios

* increase bottom bar opacity to account for hdr

* remove unused import

* fix live photo play button not updating

* fix map marker -> galleryviewer

* remove video_player

* fix hdr playback on android

* fix looping

* remove unused dependencies

* update to latest player commit

* fix player controls hiding when video is not playing

* fix restart video

* stop showing motion video after ending when looping is disabled

* delay video initialization to avoid placeholder flicker

* faster animation

* shorter delay

* small delay for image -> video on android

* fix: lint

* hide stacked children when controls are hidden, avoid bottom bar dropping

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: shenlong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
  • Loading branch information
4 people authored Dec 4, 2024
1 parent 5060ee9 commit 3c38851
Show file tree
Hide file tree
Showing 44 changed files with 1,621 additions and 1,239 deletions.
4 changes: 2 additions & 2 deletions mobile/android/app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ if (keystorePropertiesFile.exists()) {
}

android {
compileSdkVersion 34
compileSdkVersion 35

compileOptions {
sourceCompatibility JavaVersion.VERSION_17
Expand All @@ -47,7 +47,7 @@ android {
defaultConfig {
applicationId "app.alextran.immich"
minSdkVersion 26
targetSdkVersion 34
targetSdkVersion 35
versionCode flutterVersionCode.toInteger()
versionName flutterVersionName
}
Expand Down
2 changes: 1 addition & 1 deletion mobile/android/app/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@

<meta-data
android:name="io.flutter.embedding.android.EnableImpeller"
android:value="false" />
android:value="true" />

<meta-data
android:name="com.google.firebase.messaging.default_notification_icon"
Expand Down
4 changes: 2 additions & 2 deletions mobile/android/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ subprojects {
if (project.plugins.hasPlugin("com.android.application") ||
project.plugins.hasPlugin("com.android.library")) {
project.android {
compileSdkVersion 34
buildToolsVersion "34.0.0"
compileSdkVersion 35
buildToolsVersion "35.0.0"
}
}
}
Expand Down
19 changes: 9 additions & 10 deletions mobile/ios/Podfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ PODS:
- maplibre_gl (0.0.1):
- Flutter
- MapLibre (= 5.14.0-pre3)
- native_video_player (1.0.0):
- Flutter
- package_info_plus (0.4.5):
- Flutter
- path_provider_foundation (0.0.1):
Expand Down Expand Up @@ -93,9 +95,6 @@ PODS:
- Toast (4.0.0)
- url_launcher_ios (0.0.1):
- Flutter
- video_player_avfoundation (0.0.1):
- Flutter
- FlutterMacOS
- wakelock_plus (0.0.1):
- Flutter

Expand All @@ -115,6 +114,7 @@ DEPENDENCIES:
- integration_test (from `.symlinks/plugins/integration_test/ios`)
- isar_flutter_libs (from `.symlinks/plugins/isar_flutter_libs/ios`)
- maplibre_gl (from `.symlinks/plugins/maplibre_gl/ios`)
- native_video_player (from `.symlinks/plugins/native_video_player/ios`)
- package_info_plus (from `.symlinks/plugins/package_info_plus/ios`)
- path_provider_foundation (from `.symlinks/plugins/path_provider_foundation/darwin`)
- path_provider_ios (from `.symlinks/plugins/path_provider_ios/ios`)
Expand All @@ -124,7 +124,6 @@ DEPENDENCIES:
- shared_preferences_foundation (from `.symlinks/plugins/shared_preferences_foundation/darwin`)
- sqflite (from `.symlinks/plugins/sqflite/darwin`)
- url_launcher_ios (from `.symlinks/plugins/url_launcher_ios/ios`)
- video_player_avfoundation (from `.symlinks/plugins/video_player_avfoundation/darwin`)
- wakelock_plus (from `.symlinks/plugins/wakelock_plus/ios`)

SPEC REPOS:
Expand Down Expand Up @@ -168,6 +167,8 @@ EXTERNAL SOURCES:
:path: ".symlinks/plugins/isar_flutter_libs/ios"
maplibre_gl:
:path: ".symlinks/plugins/maplibre_gl/ios"
native_video_player:
:path: ".symlinks/plugins/native_video_player/ios"
package_info_plus:
:path: ".symlinks/plugins/package_info_plus/ios"
path_provider_foundation:
Expand All @@ -186,15 +187,13 @@ EXTERNAL SOURCES:
:path: ".symlinks/plugins/sqflite/darwin"
url_launcher_ios:
:path: ".symlinks/plugins/url_launcher_ios/ios"
video_player_avfoundation:
:path: ".symlinks/plugins/video_player_avfoundation/darwin"
wakelock_plus:
:path: ".symlinks/plugins/wakelock_plus/ios"

SPEC CHECKSUMS:
background_downloader: 9f788ffc5de45acf87d6380e91ca0841066c18cf
connectivity_plus: ddd7f30999e1faaef5967c23d5b6d503d10434db
device_info_plus: 97af1d7e84681a90d0693e63169a5d50e0839a0d
device_info_plus: bf2e3232933866d73fe290f2942f2156cdd10342
DKImagePickerController: 946cec48c7873164274ecc4624d19e3da4c1ef3c
DKPhotoGallery: b3834fecb755ee09a593d7c9e389d8b5d6deed60
file_picker: 09aa5ec1ab24135ccd7a1621c46c84134bfd6655
Expand All @@ -210,20 +209,20 @@ SPEC CHECKSUMS:
isar_flutter_libs: fdf730ca925d05687f36d7f1d355e482529ed097
MapLibre: 620fc933c1d6029b33738c905c1490d024e5d4ef
maplibre_gl: a2efec727dd340e4c65e26d2b03b584f14881fd9
package_info_plus: 58f0028419748fad15bf008b270aaa8e54380b1c
native_video_player: d12af78a1a4a8cf09775a5177d5b392def6fd23c
package_info_plus: c0502532a26c7662a62a356cebe2692ec5fe4ec4
path_provider_foundation: 2b6b4c569c0fb62ec74538f866245ac84301af46
path_provider_ios: 14f3d2fd28c4fdb42f44e0f751d12861c43cee02
permission_handler_apple: 9878588469a2b0d0fc1e048d9f43605f92e6cec2
photo_manager: ff695c7a1dd5bc379974953a2b5c0a293f7c4c8a
SAMKeychain: 483e1c9f32984d50ca961e26818a534283b4cd5c
SDWebImage: 066c47b573f408f18caa467d71deace7c0f8280d
share_plus: 8875f4f2500512ea181eef553c3e27dba5135aad
share_plus: 8b6f8b3447e494cca5317c8c3073de39b3600d1f
shared_preferences_foundation: fcdcbc04712aee1108ac7fda236f363274528f78
sqflite: 673a0e54cc04b7d6dba8d24fb8095b31c3a99eec
SwiftyGif: 706c60cf65fa2bc5ee0313beece843c8eb8194d4
Toast: 91b396c56ee72a5790816f40d3a94dd357abc196
url_launcher_ios: 5334b05cef931de560670eeae103fd3e431ac3fe
video_player_avfoundation: 7c6c11d8470e1675df7397027218274b6d2360b3
wakelock_plus: 78ec7c5b202cab7761af8e2b2b3d0671be6c4ae1

PODFILE CHECKSUM: 64c9b5291666c0ca3caabdfe9865c141ac40321d
Expand Down
75 changes: 43 additions & 32 deletions mobile/ios/Runner/AppDelegate.swift
Original file line number Diff line number Diff line change
@@ -1,48 +1,59 @@
import UIKit
import shared_preferences_foundation
import Flutter
import BackgroundTasks
import Flutter
import UIKit
import path_provider_ios
import photo_manager
import permission_handler_apple
import photo_manager
import shared_preferences_foundation

@main
@objc class AppDelegate: FlutterAppDelegate {

override func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
) -> Bool {

// Required for flutter_local_notification
if #available(iOS 10.0, *) {
UNUserNotificationCenter.current().delegate = self as? UNUserNotificationCenterDelegate
// Required for flutter_local_notification
if #available(iOS 10.0, *) {
UNUserNotificationCenter.current().delegate = self as? UNUserNotificationCenterDelegate
}

do {
try AVAudioSession.sharedInstance().setCategory(.playback, mode: .default)
try AVAudioSession.sharedInstance().setActive(true)
} catch {
print("Failed to set audio session category. Error: \(error)")
}

GeneratedPluginRegistrant.register(with: self)
BackgroundServicePlugin.registerBackgroundProcessing()

BackgroundServicePlugin.register(with: self.registrar(forPlugin: "BackgroundServicePlugin")!)

BackgroundServicePlugin.setPluginRegistrantCallback { registry in
if !registry.hasPlugin("org.cocoapods.path-provider-ios") {
FLTPathProviderPlugin.register(
with: registry.registrar(forPlugin: "org.cocoapods.path-provider-ios")!)
}

if !registry.hasPlugin("org.cocoapods.photo-manager") {
PhotoManagerPlugin.register(
with: registry.registrar(forPlugin: "org.cocoapods.photo-manager")!)
}

GeneratedPluginRegistrant.register(with: self)
BackgroundServicePlugin.registerBackgroundProcessing()

BackgroundServicePlugin.register(with: self.registrar(forPlugin: "BackgroundServicePlugin")!)

BackgroundServicePlugin.setPluginRegistrantCallback { registry in
if !registry.hasPlugin("org.cocoapods.path-provider-ios") {
FLTPathProviderPlugin.register(with: registry.registrar(forPlugin: "org.cocoapods.path-provider-ios")!)
}

if !registry.hasPlugin("org.cocoapods.photo-manager") {
PhotoManagerPlugin.register(with: registry.registrar(forPlugin: "org.cocoapods.photo-manager")!)
}

if !registry.hasPlugin("org.cocoapods.shared-preferences-foundation") {
SharedPreferencesPlugin.register(with: registry.registrar(forPlugin: "org.cocoapods.shared-preferences-foundation")!)
}

if !registry.hasPlugin("org.cocoapods.permission-handler-apple") {
PermissionHandlerPlugin.register(with: registry.registrar(forPlugin: "org.cocoapods.permission-handler-apple")!)
}
if !registry.hasPlugin("org.cocoapods.shared-preferences-foundation") {
SharedPreferencesPlugin.register(
with: registry.registrar(forPlugin: "org.cocoapods.shared-preferences-foundation")!)
}

return super.application(application, didFinishLaunchingWithOptions: launchOptions)

if !registry.hasPlugin("org.cocoapods.permission-handler-apple") {
PermissionHandlerPlugin.register(
with: registry.registrar(forPlugin: "org.cocoapods.permission-handler-apple")!)
}
}

return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}

}
2 changes: 1 addition & 1 deletion mobile/lib/constants/immich_colors.dart
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ const String defaultColorPresetName = "indigo";
const Color immichBrandColorLight = Color(0xFF4150AF);
const Color immichBrandColorDark = Color(0xFFACCBFA);
const Color whiteOpacity75 = Color.fromARGB((0.75 * 255) ~/ 1, 255, 255, 255);
const Color blackOpacity90 = Color.fromARGB((0.90 * 255) ~/ 1, 0, 0, 0);
const Color red400 = Color(0xFFEF5350);
const Color grey200 = Color(0xFFEEEEEE);

final Map<ImmichColorPreset, ImmichTheme> _themePresetsMap = {
ImmichColorPreset.indigo: ImmichTheme(
Expand Down
115 changes: 88 additions & 27 deletions mobile/lib/entities/asset.entity.dart
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import 'dart:convert';
import 'dart:io';

import 'package:immich_mobile/entities/exif_info.entity.dart';
import 'package:immich_mobile/utils/hash.dart';
Expand All @@ -22,12 +23,8 @@ class Asset {
durationInSeconds = remote.duration.toDuration()?.inSeconds ?? 0,
type = remote.type.toAssetType(),
fileName = remote.originalFileName,
height = isFlipped(remote)
? remote.exifInfo?.exifImageWidth?.toInt()
: remote.exifInfo?.exifImageHeight?.toInt(),
width = isFlipped(remote)
? remote.exifInfo?.exifImageHeight?.toInt()
: remote.exifInfo?.exifImageWidth?.toInt(),
height = remote.exifInfo?.exifImageHeight?.toInt(),
width = remote.exifInfo?.exifImageWidth?.toInt(),
livePhotoVideoId = remote.livePhotoVideoId,
ownerId = fastHash(remote.ownerId),
exifInfo =
Expand Down Expand Up @@ -93,6 +90,27 @@ class Asset {

set local(AssetEntity? assetEntity) => _local = assetEntity;

@ignore
bool _didUpdateLocal = false;

@ignore
Future<AssetEntity> get localAsync async {
final local = this.local;
if (local == null) {
throw Exception('Asset $fileName has no local data');
}

final updatedLocal =
_didUpdateLocal ? local : await local.obtainForNewProperties();
if (updatedLocal == null) {
throw Exception('Could not fetch local data for $fileName');
}

this.local = updatedLocal;
_didUpdateLocal = true;
return updatedLocal;
}

Id id = Isar.autoIncrement;

/// stores the raw SHA1 bytes as a base64 String
Expand Down Expand Up @@ -150,10 +168,21 @@ class Asset {

int stackCount;

/// Aspect ratio of the asset
/// Returns null if the asset has no sync access to the exif info
@ignore
double? get aspectRatio =>
width == null || height == null ? 0 : width! / height!;
double? get aspectRatio {
final orientatedWidth = this.orientatedWidth;
final orientatedHeight = this.orientatedHeight;

if (orientatedWidth != null &&
orientatedHeight != null &&
orientatedWidth > 0 &&
orientatedHeight > 0) {
return orientatedWidth.toDouble() / orientatedHeight.toDouble();
}

return null;
}

/// `true` if this [Asset] is present on the device
@ignore
Expand All @@ -172,6 +201,12 @@ class Asset {
@ignore
bool get isImage => type == AssetType.image;

@ignore
bool get isVideo => type == AssetType.video;

@ignore
bool get isMotionPhoto => livePhotoVideoId != null;

@ignore
AssetState get storage {
if (isRemote && isLocal) {
Expand All @@ -192,6 +227,50 @@ class Asset {
@ignore
set byteHash(List<int> hash) => checksum = base64.encode(hash);

/// Returns null if the asset has no sync access to the exif info
@ignore
@pragma('vm:prefer-inline')
bool? get isFlipped {
final exifInfo = this.exifInfo;
if (exifInfo != null) {
return exifInfo.isFlipped;
}

if (_didUpdateLocal && Platform.isAndroid) {
final local = this.local;
if (local == null) {
throw Exception('Asset $fileName has no local data');
}
return local.orientation == 90 || local.orientation == 270;
}

return null;
}

/// Returns null if the asset has no sync access to the exif info
@ignore
@pragma('vm:prefer-inline')
int? get orientatedHeight {
final isFlipped = this.isFlipped;
if (isFlipped == null) {
return null;
}

return isFlipped ? width : height;
}

/// Returns null if the asset has no sync access to the exif info
@ignore
@pragma('vm:prefer-inline')
int? get orientatedWidth {
final isFlipped = this.isFlipped;
if (isFlipped == null) {
return null;
}

return isFlipped ? height : width;
}

@override
bool operator ==(other) {
if (other is! Asset) return false;
Expand Down Expand Up @@ -511,21 +590,3 @@ extension AssetsHelper on IsarCollection<Asset> {
return where().anyOf(ids, (q, String e) => q.localIdEqualTo(e));
}
}

/// Returns `true` if this [int] is flipped 90° clockwise
bool isRotated90CW(int orientation) {
return [7, 8, -90].contains(orientation);
}

/// Returns `true` if this [int] is flipped 270° clockwise
bool isRotated270CW(int orientation) {
return [5, 6, 90].contains(orientation);
}

/// Returns `true` if this [Asset] is flipped 90° or 270° clockwise
bool isFlipped(AssetResponseDto response) {
final int orientation =
int.tryParse(response.exifInfo?.orientation ?? '0') ?? 0;
return orientation != 0 &&
(isRotated90CW(orientation) || isRotated270CW(orientation));
}
Loading

0 comments on commit 3c38851

Please sign in to comment.