Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QA] files uploaded via s3cmd with a prefix are not seen by occ file:scan #971

Open
jnweiger opened this issue Aug 19, 2022 · 1 comment

Comments

@jnweiger
Copy link
Contributor

jnweiger commented Aug 19, 2022

Seen while testing files_external_s3-2.0.0-rc.1 with owncloud core 10.11.0-alpha.2

  • Configure an external storage using a scality s3 server
    image

  • Check number of objects with occ file:scan --all
    +---------+-------+--------------+------------------+
    | Folders | Files | Elapsed time | Items per second |
    +---------+-------+--------------+------------------+
    | 24      | 31    | 00:00:01     | 56               |
    +---------+-------+--------------+------------------+
    
    
  • Add a file to the bucket (without path prefix) s3cmd put /var/www/owncloud/data/owncloud.log s3://oc-external/
  • Check number of objects with occ file:scan --all -- it is one more. OK
    +---------+-------+--------------+------------------+
    | Folders | Files | Elapsed time | Items per second |
    +---------+-------+--------------+------------------+
    | 24      | 32    | 00:00:01     | 54               |
    +---------+-------+--------------+------------------+
    
    
  • Add a file to the bucket (with prefix) s3cmd put /var/www/owncloud/data/owncloud.log s3://oc-external/logs/oc.log
  • Check number of objects with occ file:scan --all -- nothing changes. BAD
    +---------+-------+--------------+------------------+
    | Folders | Files | Elapsed time | Items per second |
    +---------+-------+--------------+------------------+
    | 24      | 32    | 00:00:01     | 49               |
    +---------+-------+--------------+------------------+
    
    
  • Visit the external storage via owncloud. The /oc-bucket/logs folder does not appear. BAD

Workaround:

  • Create the /oc-bucket/logs folder via owncloud.
    image

  • Examine contents of the logs folder. The oc.log file is there.
    image

  • Now also occ file:scan reports the new folder and the new file.
@jnweiger
Copy link
Contributor Author

Note: other external storages, likes e.g. SFTP do not have this problem. It seems specific to s3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant