Modifications for large archives; other minor changes #380

eliotmcintire · 2023-11-29T18:39:24Z

No description provided.

… rm dirs

They may fail when internet connection is flaky. From experience, more than 20 minutes downloads are probably stalling. However, we may pass this as a user parameter in reproducible options (not PR'ed).

…orLargeArchives

@eliotmcintire

@eliotmcintire see TODOs re: tests where it's not obvious to me what the correct behaviour should be

achubaty · 2024-05-09T04:40:17Z

I've worked through many of the check failures but there are still outstanding issues blocking merge:

possibly unsafe calls: assignInNamespace on branch modsForLargeArchives #382
outstanding local test failures for modsForLargeArchives branch #387
multiple warnings in checks on modsForLargeArchives branch - file exists but no db entry #388

allow updating (i.e., needed for `.message$PreProcessIndent`) without using `assignInNamespace`, which cannot be used in packages (#382)

some of these are already used in `SpaDES.core`

)

… entry (#387)

add http-user-agent

try any::RCurl to work around R-devel on Windows trying to install source version and failing

try devel version of pak with R CMD Check workflows to help with R-devel failures

This reverts commit ec96884.

'/' is illegal character for windows artifact uploads for GHA

`RApiSerialize' update requires rebuild/reinstall of `qs` from source

eliotmcintire and others added 30 commits November 24, 2023 12:30

now that fs is Imports, use it more (where faster/better)

f13e5fd

rm system.time

40bee2b

allow targetFile = NA ... meaning no expected targetFile

5ab3e2c

prevent 2x Checksuming same files

4f73510

don't linkOrCopy at end if files are already same

39c938e

prevent false need to re-checksum because of "similar"

2ea5b20

give additional message when Checksums failing-- include inputPaths

31ca869

base unzip can handle bigger (set and tested up to >5e9)

6b1e547

bugfixes for archive file lists -- need to switch from \\ to / paths,…

b93bdcd

… rm dirs

prevent unnecessary redos

c8c1e2a

messaging updates

2448be9

linkOrCopyUpdateOnly

3ad7855

v2.0.10.9001 & bump date

efd5fe8

list2

88db9ee

bugfix targetFilePath -- when not "needed"

5a6abea

better error ( not ee$destinationPath)

a9278f9

normPath - more accurate replacement of normalizePath

d510b9e

makeRelative -- tried fs:path_rel -- not good; better gsub

341adb0

gdalResample -- rm -tap --> rounds coordinates to nearest res

9a81fd3

gdalMask -- allow gridded -- but not faster

e80b33a

message correction

166934f

change -- "match by basename" in Checksums

324113d

listNamed instead of list2

c46fd99

Build

12ae652

Trying to add a time out for big GDrive files

1f13c41

They may fail when internet connection is flaky. From experience, more than 20 minutes downloads are probably stalling. However, we may pass this as a user parameter in reproducible options (not PR'ed).

listNamed docs

657ca70

Merge remote-tracking branch 'origin/modsForLargeArchives' into modsF…

178f575

…orLargeArchives

withTimeout

cf05292

change setup.R for 585

fa48eb9

modify ... done spacing

1eacd37

achubaty added 7 commits May 8, 2024 20:02

fix relativeToWhat test

7d50d04

tweak to prev

fd9867e

fix cache test failures

3159cb8

[skip-ci] misc cleanup

4b627ff

[WIP] fixing cache helper tests

9b8b182

@eliotmcintire see TODOs re: tests where it's not obvious to me what the correct behaviour should be

[WIP] fixing prepInputs test failures + cleanup

800d6b8

[skip-ci] add cache test comment + minor cleanup

041b470

achubaty added 21 commits May 9, 2024 11:53

move all internal .messages to an internal environment

19a20b7

allow updating (i.e., needed for `.message$PreProcessIndent`) without using `assignInNamespace`, which cannot be used in packages (#382)

move SysInfo to .pkgEnv

f1ba264

reorganize exportedMethods.R

e7098fb

export message*() for use with other packages

09a6f82

some of these are already used in `SpaDES.core`

[skip-ci] message docs formatting

97e9e5c

[skip-ci] add .reproEnv to package env docs

a1f74e5

minor docs tweak

5f00871

[v2.0.12.9010] fix clearCache + cleanup cache after each test (#388)

8cd5302

put cacheSaveFormat option back in list to keep/check in msic test (#387

36b4d1c

)

[skip-ci] minor whitespace/formatting

4e250c9

[v2.0.12.9011] adjust showSimilar tests to expect most recent cache…

52f4d36

… entry (#387)

cleanup + remove faulty windows test

0a8e888

skip a test on R < 4.3

c235fc3

tweak GHA check workflow

5979707

add http-user-agent

Update R-CMD-check.yaml

ec96884

try any::RCurl to work around R-devel on Windows trying to install source version and failing

Update R-CMD-check.yaml

6b05b37

try devel version of pak with R CMD Check workflows to help with R-devel failures

[skip-ci] minor

49650eb

Revert "Update R-CMD-check.yaml"

bf5db25

This reverts commit ec96884.

GHA use '-' instead of '/' for oldrel specification

510d7b8

'/' is illegal character for windows artifact uploads for GHA

GHA: try install qs from GitHub to force source install

a1221d5

`RApiSerialize' update requires rebuild/reinstall of `qs` from source

[skip-ci] revdeps checks

b6b6490

achubaty merged commit 0ecd145 into development May 15, 2024
15 of 17 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modifications for large archives; other minor changes #380

Modifications for large archives; other minor changes #380

eliotmcintire commented Nov 29, 2023

achubaty commented May 9, 2024

Modifications for large archives; other minor changes #380

Modifications for large archives; other minor changes #380

Conversation

eliotmcintire commented Nov 29, 2023

achubaty commented May 9, 2024