Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] ffigen tries to write to working-directory #494

Closed
thomas725 opened this issue Jun 7, 2022 · 19 comments
Closed

[Bug] ffigen tries to write to working-directory #494

thomas725 opened this issue Jun 7, 2022 · 19 comments
Labels
bug Something isn't working wontfix This will not be worked on

Comments

@thomas725
Copy link
Contributor

Describe the bug

I need to run flutter_rust_bridge_codegen with working directory = / (root folder) to work around http://cjycode.com/flutter_rust_bridge/troubleshooting.html?highlight=std#the-generated-store_dart_post_cobject-has-the-wrong-signature--stdargh-file-not-found-in-linux #108 #472

But it seems in some cases (api.rs pub struct with pub const in impl block), flutter_rust_bridge_codegen tries to write to the working directory, which results in a permission denied error.

Codegen logs with RUST_LOG=debug env variable

$RUST_LOG=debug ; REPO_DIR="$PWD" ; cd / ; flutter_rust_bridge_codegen \
  --rust-input "$REPO_DIR/native/src/api.rs" \
  --dart-output "$REPO_DIR/lib/bridge_generated.dart" \
  --c-output "$REPO_DIR/ios/Runner/bridge_generated.h"
[2022-06-07T05:26:03Z INFO  lib_flutter_rust_bridge_codegen] Picked config: Opts { rust_input_path: "/home/thomas/Development/git/flutter-playground/native/src/api.rs", dart_output_path: "/home/thomas/Development/git/flutter-playground/lib/bridge_generated.dart", dart_decl_output_path: None, c_output_path: ["/home/thomas/Development/git/flutter-playground/ios/Runner/bridge_generated.h"], rust_crate_dir: "/home/thomas/Development/git/flutter-playground/native", rust_output_path: "/home/thomas/Development/git/flutter-playground/native/src/bridge_generated.rs", class_name: "Native", dart_format_line_length: 80, skip_add_mod_to_lib: false, llvm_path: ["/opt/homebrew/opt/llvm", "/usr/local/opt/llvm", "/usr/lib/llvm-9", "/usr/lib/llvm-10", "/usr/lib/llvm-11", "/usr/lib/llvm-12", "/usr/lib/llvm-13", "/usr/lib/llvm-14", "/usr/lib/", "/usr/lib64/", "C:/Program Files/llvm", "C:/Program Files/LLVM", "C:/msys64/mingw64"], llvm_compiler_opts: "", manifest_path: "/home/thomas/Development/git/flutter-playground/native/Cargo.toml", dart_root: Some("/home/thomas/Development/git/flutter-playground"), build_runner: true }
[2022-06-07T05:26:03Z INFO  lib_flutter_rust_bridge_codegen] Phase: Parse source code to AST
[2022-06-07T05:26:03Z INFO  lib_flutter_rust_bridge_codegen] Phase: Parse AST to IR
[2022-06-07T05:26:03Z INFO  lib_flutter_rust_bridge_codegen] Phase: Transform IR
[2022-06-07T05:26:03Z INFO  lib_flutter_rust_bridge_codegen] Phase: Generate Rust code
[2022-06-07T05:26:03Z INFO  lib_flutter_rust_bridge_codegen] Phase: Generate Dart code
[2022-06-07T05:26:03Z INFO  lib_flutter_rust_bridge_codegen] Phase: Other things
[2022-06-07T05:26:03Z WARN  cbindgen::bindgen::parser] Skip native::ONE_SECOND - (not `pub`).
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::LogEntry - opaque (Struct is not marked #[repr(C)] or #[repr(transparent)].).
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::LogEntry::LEVEL_TRACE.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::LogEntry::LEVEL_DEBUG.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::LogEntry::LEVEL_INFO.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::LogEntry::LEVEL_WARN.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::LogEntry::LEVEL_ERROR.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::wire_rust_release_mode.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::wire_environment_info.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::wire_tick.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::wire_create_log_stream.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::wire_rust_set_up.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::free_WireSyncReturnStruct.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::DartPort.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::DartPostCObjectFnType.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::store_dart_post_cobject.
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::WireSyncReturnStruct.
[2022-06-07T05:26:03Z WARN  cbindgen::bindgen::parser] Skip native::INIT_LOGGER_ONCE - (not `pub`).
[2022-06-07T05:26:03Z WARN  cbindgen::bindgen::parser] Skip native::INIT_LOGGER_ONCE - (not `no_mangle`).
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::SendToDartLogger - opaque (Struct is not marked #[repr(C)] or #[repr(transparent)].).
[2022-06-07T05:26:03Z INFO  cbindgen::bindgen::parser] Take native::MyMobileLogger - opaque (Struct is not marked #[repr(C)] or #[repr(transparent)].).
[2022-06-07T05:26:03Z WARN  cbindgen::bindgen::parser] Skip native::LEVEL_TRACE - (not `pub`).
[2022-06-07T05:26:04Z WARN  lib_flutter_rust_bridge_codegen::commands] command="sh" "-c" "dart pub global run ffigen --config \"/tmp/.tmpUsPtj3\"" stdout=Running in Directory: '/'
    Input Headers: [/tmp/.tmpExG4j4.h]
     stderr=Unhandled exception:
    FileSystemException: Cannot create file, path = 'temp_for_macros.hpp' (OS Error: Permission denied, errno = 13)
    #0      _File.throwIfError (dart:io/file_impl.dart:635:7)
    dart-lang/ffigen#1      _File.createSync (dart:io/file_impl.dart:273:5)
    dart-lang/ffigen#2      createFileForMacros (package:ffigen/src/header_parser/sub_parsers/macro_parser.dart:188:8)
    dart-lang/ffigen#3      parseSavedMacros (package:ffigen/src/header_parser/sub_parsers/macro_parser.dart:59:16)
    dart-lang/ffigen#4      parseToBindings (package:ffigen/src/header_parser/parser.dart:111:19)
    dart-lang/ffigen#5      parse (package:ffigen/src/header_parser/parser.dart:24:20)
    dart-lang/ffigen#6      main (package:ffigen/src/executables/ffigen.dart:54:19)
    dart-lang/ffigen#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:295:32)
    dart-lang/ffigen#8      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)
    
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ffigen failed:
stderr: Unhandled exception:
FileSystemException: Cannot create file, path = 'temp_for_macros.hpp' (OS Error: Permission denied, errno = 13)
#0      _File.throwIfError (dart:io/file_impl.dart:635:7)
dart-lang/ffigen#1      _File.createSync (dart:io/file_impl.dart:273:5)
dart-lang/ffigen#2      createFileForMacros (package:ffigen/src/header_parser/sub_parsers/macro_parser.dart:188:8)
dart-lang/ffigen#3      parseSavedMacros (package:ffigen/src/header_parser/sub_parsers/macro_parser.dart:59:16)
dart-lang/ffigen#4      parseToBindings (package:ffigen/src/header_parser/parser.dart:111:19)
dart-lang/ffigen#5      parse (package:ffigen/src/header_parser/parser.dart:24:20)
dart-lang/ffigen#6      main (package:ffigen/src/executables/ffigen.dart:54:19)
dart-lang/ffigen#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:295:32)
dart-lang/ffigen#8      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

stdout: Running in Directory: '/'
Input Headers: [/tmp/.tmpExG4j4.h]
', /home/thomas/.cargo/registry/src/github.com-1ecc6299db9ec823/flutter_rust_bridge_codegen-1.30.0/src/main.rs:16:23
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

To Reproduce

No response

Expected behavior

No response

Generated binding code

No response

OS

Manjaro (~Arch) Linux

Version of flutter_rust_bridge_codegen

1.30.0

Flutter info

[✓] Flutter (Channel stable, 3.0.1, on Manjaro Linux 5.15.41-1-MANJARO, locale en_US.UTF-8)
    • Flutter version 3.0.1 at /opt/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision fb57da5f94 (3 weeks ago), 2022-05-19 15:50:29 -0700
    • Engine revision caaafc5604
    • Dart version 2.17.1
    • DevTools version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /home/thomas/Android/Sdk
    • Platform android-31, build-tools 30.0.3
    • ANDROID_HOME = /home/thomas/Android/Sdk
    • Java binary at: /opt/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • CHROME_EXECUTABLE = chromium

[✓] Linux toolchain - develop for Linux desktop
    • clang version 13.0.1
    • cmake version 3.23.1
    • ninja version 1.10.2
    • pkg-config version 1.8.0

[✓] Android Studio (version 2021.2)
    • Android Studio at /opt/android-studio
    • Flutter plugin version 64.1.2
    • Dart plugin version 212.5744
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] Connected device (2 available)
    • Linux (desktop) • linux  • linux-x64      • Manjaro Linux 5.15.41-1-MANJARO
    • Chrome (web)    • chrome • web-javascript • Chromium 102.0.5005.61 Arch Linux

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

Version of clang++

No response

Version of ffigen

No response

Additional context

see also: #486

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 7, 2022

flutter_rust_bridge is doing nothing but invoking ffigen. So could you please try to invoke ffigen directly, without such indirection from flutter_rust_bridge, and see what happens

@thomas725
Copy link
Contributor Author

Can you give me some pointers on how to "invoke ffigen directly"?
Sorry, newbie here, I got no idea what ffigen even is, but I guess I'll have to read up on it a bit...

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 7, 2022

Just run commands in a shell...

@thomas725
Copy link
Contributor Author

Well yeah obviously, but which command with which parameters?

At the moment I'm using just which has been setup by https://github.com/Desdaemon/flutter_rust_bridge_template

Of course it's very easy to see what get's called by just by looking at the .justfile in the repository, which is how I got to executing this in a terminal as documented above:

RUST_LOG=debug ; REPO_DIR="$PWD" ; cd / ; flutter_rust_bridge_codegen \
  --rust-input "$REPO_DIR/native/src/api.rs" \
  --dart-output "$REPO_DIR/lib/bridge_generated.dart" \
  --c-output "$REPO_DIR/ios/Runner/bridge_generated.h"

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 7, 2022

[2022-06-07T05:26:04Z WARN lib_flutter_rust_bridge_codegen::commands] command="sh" "-c" "dart pub global run ffigen --config "/tmp/.tmpUsPtj3"" stdout=Running in Directory: '/'

Thus I guess: dart pub global run ffigen --config "/tmp/.tmpUsPtj3"

Well indeed you also need to put contents into that auto-generated temporary config file.

@thomas725
Copy link
Contributor Author

thomas725 commented Jun 7, 2022

And what content's should I put there? The config file created by flutter_rust_bridge in the /tmp/ folder get's deleted instantly so I don't know what it's content should be. Without specifying a config file I get this output:

$ dart pub global run ffigen                                                                                                                                               ✔ 
Running in Directory: '/home/thomas/Development/git/flutter-playground'
[SEVERE] : Couldn't find an entry for 'ffigen' in pubspec.yaml.

UPDATE: so by googling a bit I found I can watch the /tmp/ directory for usage of files matching a pattern line beginning with a dot like so, including the result of letting the flutter_rust_bridge run:

$ inotifywait -e modify,create,delete --monitor --include "\..*" "/tmp/"
Setting up watches.
Watches established.
/tmp/ CREATE .tmpnev1k7
/tmp/ CREATE .tmpNU0Hcz.h
/tmp/ MODIFY .tmpNU0Hcz.h
/tmp/ MODIFY .tmpNU0Hcz.h
/tmp/ CREATE .tmp1cIan2
/tmp/ MODIFY .tmp1cIan2
/tmp/ DELETE .tmp1cIan2
/tmp/ DELETE .tmpNU0Hcz.h
/tmp/ DELETE .tmpnev1k7
/tmp/ MODIFY gitstatus.POWERLEVEL9K.1000.56343.1654623323.1.fifo

So I guess my next step is to process this output and copy every line that contains the String CREATE or MODIFY and copy away the resulting file for manual inspection afterwards. Tomorrow. Now I'm tired and I gotta sleep.

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 7, 2022

And what content's should I put there? The config file created by flutter_rust_bridge in the /tmp/ folder get's deleted instantly so I don't know what it's content should be.

Well, maybe modify frb_codegen and force it not to delete it... Or use your method of "copying it before it is deleted" which sounds promising!

Third method, if I remember correctly, when you enable RUST_LOG=debug, the config prints out automatically. (Notice your RUST_LOG=debug is wrong in the first post in this issue)

@thomas725
Copy link
Contributor Author

thomas725 commented Jun 8, 2022

I've set RUST_LOG=debug when I created this issue and posted the output, so either I've done it wrong or in it's current version it doesn't make it print the config file's content.

But now I've came up with this bash-script to copy those temp files away before deletion:

target="$1"

cd "$target"
mkdir backup/

inotifywait -e modify,create,delete --monitor --include "\..*" "$target" | \
while read line
do
  echo "$line"
  if [[ "$line" == "$target CREATE "* ]] || [[ "$line" == "$target MODIFY "* ]]
  then
    filename=${line#"$target CREATE "}
    filename=${filename#"$target MODIFY "}
    cp "$filename" backup/
  fi
done

which I was able to use to copy the config file's content away, which is:

        output: '/tmp/.tmpXRLVsm'
        name: 'NativeWire'
        description: 'generated by flutter_rust_bridge'
        headers:
          entry-points:
            - '/tmp/.tmphNxzbQ.h'
          include-directives:
            - '/tmp/.tmphNxzbQ.h'
        comments: false
        preamble: |
          // ignore_for_file: camel_case_types, non_constant_identifier_names, avoid_positional_boolean_parameters, annotate_overrides, constant_identifier_names
        
        llvm-path:
           - '/opt/homebrew/opt/llvm'
           - '/usr/local/opt/llvm'
           - '/usr/lib/llvm-9'
           - '/usr/lib/llvm-10'
           - '/usr/lib/llvm-11'
           - '/usr/lib/llvm-12'
           - '/usr/lib/llvm-13'
           - '/usr/lib/llvm-14'
           - '/usr/lib/'
           - '/usr/lib64/'
           - 'C:/Program Files/llvm'
           - 'C:/Program Files/LLVM'
           - 'C:/msys64/mingw64'

After replacing the path of the deleted file /tmp/.tmphNxzbQ.h with my backup in /tmp/backup/.tmphNxzbQ.h in that config, running ffigen manually shows the same problem. When running from / = rootdirectory:

$ dart pub global run ffigen --config "/tmp/backup/.tmpWFhbLL"
Running in Directory: '/'
Input Headers: [/tmp/backup/.tmphNxzbQ.h]
Unhandled exception:
FileSystemException: Cannot create file, path = 'temp_for_macros.hpp' (OS Error: Permission denied, errno = 13)
#0      _File.throwIfError (dart:io/file_impl.dart:635:7)
dart-lang/ffigen#1      _File.createSync (dart:io/file_impl.dart:273:5)
dart-lang/ffigen#2      createFileForMacros (package:ffigen/src/header_parser/sub_parsers/macro_parser.dart:188:8)
dart-lang/ffigen#3      parseSavedMacros (package:ffigen/src/header_parser/sub_parsers/macro_parser.dart:59:16)
dart-lang/ffigen#4      parseToBindings (package:ffigen/src/header_parser/parser.dart:111:19)
dart-lang/ffigen#5      parse (package:ffigen/src/header_parser/parser.dart:24:20)
dart-lang/ffigen#6      main (package:ffigen/src/executables/ffigen.dart:54:19)
dart-lang/ffigen#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:295:32)
dart-lang/ffigen#8      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

Instead running ffigen from my repository path (where it has write access) gives:

$ dart pub global run ffigen --config "/tmp/backup/.tmpWFhbLL"
Running in Directory: '/home/thomas/Development/git/flutter-playground'
Input Headers: [/tmp/backup/.tmphNxzbQ.h]
[SEVERE] : Header /tmp/backup/.tmphNxzbQ.h: Total errors/warnings: 1.
[SEVERE] :     /tmp/backup/.tmphNxzbQ.h:1:10: fatal error: 'stdbool.h' file not found [Lexical or Preprocessor Issue]
Finished, Bindings generated in /tmp/backup/.tmpXRLVsm

The content of the header file that makes ffigen produce that error:

#include <stdbool.h>
#include <stdint.h>
#include <stdlib.h>

#define LogEntry_LEVEL_TRACE 5000

#define LogEntry_LEVEL_DEBUG 10000

#define LogEntry_LEVEL_INFO 20000

#define LogEntry_LEVEL_WARN 30000

#define LogEntry_LEVEL_ERROR 40000

typedef struct WireSyncReturnStruct {
  uint8_t *ptr;
  int32_t len;
  bool success;
} WireSyncReturnStruct;

typedef int64_t DartPort;

typedef bool (*DartPostCObjectFnType)(DartPort port_id, void *message);

void wire_rust_release_mode(int64_t port_);

void wire_environment_info(int64_t port_);

void wire_tick(int64_t port_);

void wire_create_log_stream(int64_t port_);

void wire_rust_set_up(int64_t port_);

void free_WireSyncReturnStruct(struct WireSyncReturnStruct val);

void store_dart_post_cobject(DartPostCObjectFnType ptr);

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 8, 2022

$RUST_LOG=debug ; REPO_DIR="$PWD" ; cd / ; flutter_rust_bridge_codegen \
  --rust-input "$REPO_DIR/native/src/api.rs" \
  --dart-output "$REPO_DIR/lib/bridge_generated.dart" \
  --c-output "$REPO_DIR/ios/Runner/bridge_generated.h"

is wrong.

RUST_LOG=debug flutter_rust_bridge_codegen something or export RUST_LOG=debug; flutter_rust_bridge_codegen something

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 8, 2022

After replacing the path of the deleted file /tmp/.tmphNxzbQ.h with my backup in /tmp/backup/.tmphNxzbQ.h in that config, running ffigen manually shows the same problem. When running from / = rootdirectory:

Great! Now it is a ffigen problem, so you have a reproducible sample and can file an issue there :)

@thomas725
Copy link
Contributor Author

thomas725 commented Jun 8, 2022

Oh, thanks for catching that mistake with setting the variables. I wonder why it works for my REPO_DIR variable.

Okey, so next I should file an issue @ https://github.com/dart-lang/ffigen/issues about ffigen trying to write to the current working directory with my given example header file, did I understand that correctly?

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 8, 2022

Okey, so next I should file an issue @ https://github.com/dart-lang/ffigen/issues about ffigen trying to write to the current working directory with my given example header file, did I understand that correctly?

Yes. I cannot fix ffigen bugs, surely :)

@thomas725 thomas725 changed the title [Bug] flutter_rust_bridge_codegen tries to write to working-directory [Bug] ffigen tries to write to working-directory Jun 8, 2022
@thomas725
Copy link
Contributor Author

Okey, so after reading some more I found a different workaround for that stdbool.h not found error @ dart-lang/native#338

export CPATH="$(clang -v 2>&1 | grep "Selected GCC installation" | rev | cut -d' ' -f1 | rev)/include"

With that I don't have to run ffigen from / but can stay at my repository path, where it's no problem when ffigen tries to write to the working directory.

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 8, 2022

You are welcomed to make a PR to documentation as well :)

@thomas725
Copy link
Contributor Author

okey, here's a documentation PR:
#497

@thomas725
Copy link
Contributor Author

does your "laugh"-smiley-reaction mean I did something stupid?

@fzyzcjy
Copy link
Owner

fzyzcjy commented Jun 8, 2022

Surely no! It means you did a good job and I smiled :)

@stale
Copy link

stale bot commented Aug 7, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Aug 7, 2022
@stale stale bot closed this as completed Aug 14, 2022
@github-actions
Copy link
Contributor

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

2 participants