Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update script to include additional functions #1

Merged
merged 1 commit into from
Nov 28, 2021

Conversation

Mechcondrid
Copy link
Contributor

update script to include the following:

  • globals for folder IDs and execution start timestamp
  • convert console output to logger output to store output for later forwarding
  • email notifications for execution start and completion with stored console log email

update script to include the following:
- globals for folder IDs and execution start timestamp
- convert console output to logger output to store output for later forwarding
- email notifications for execution start and completion with stored console log email
@dustintodd123 dustintodd123 merged commit b3b4d5c into dustintodd123:main Nov 28, 2021
@dustintodd123
Copy link
Owner

Nice fixes, not sure I fully understand the implications of using PropertiesService.getScriptProperties but I will try it out.

@Mechcondrid
Copy link
Contributor Author

well since google scripts doesn't have a conventional concept of a global variable assignment they set a "Static" variable that can be called with those functions.
as long as you don't change the property identifier it just overwrites them each run, they are unique to that single project and can't be edited outside of calling that setproperty function so they function sort of like a top level variable assignment.

that's the only way i could think of to assign a single variable that is accessible to both the function method as well as the main thread without re-defining it in each.

@Mechcondrid
Copy link
Contributor Author

i am also looking at putting in a batch or progress storage to get around the 6 minute / 30 minute execution limits.
i ran into that limit a number of times using this and had to use other methods on moves of migrated sharepoint document libraries going into google drive.
the libraries had in some cases 1000s of files so it pretty much didn't even get through half of those libraries before terminating.

@dustintodd123
Copy link
Owner

Agreed script execution timeout issue needs a fix. I couldn't come up with an elegant solution.

@Mechcondrid
Copy link
Contributor Author

Mechcondrid commented Nov 28, 2021

only way i can see is to put in a file structure (probably json or something) and record all the file and folder IDs as it progresses.
then each run of the copy function it would check to see if it had a matching entry in the file structure and skip it if it finds it then at the top of the copy function a iterative timer that both increments the file structure index and checks time passed since the execution start timestamp. it would set a recurring trigger and then break the script at 280 seconds or 1740 seconds based on a true false flag to mark if the google account is a enterprise licensed one or not.

after it hit the end of the source folder it would then have to go in and delete the triggers to keep it from repeating itself unprompted.

the total time to copy everything would increase but since we would be bypassing the execution limit it doesn't really matter.

@dustintodd123
Copy link
Owner

It's a lot of complexity to get around a limit that Google just needs to remove or adjust.

@dustintodd123
Copy link
Owner

There are some uglier hacks that could be done just to break the folder structure up into separate jobs. It would be imperfect but simpler.

@Mechcondrid
Copy link
Contributor Author

ya i agree but this way is the most fool proof that i can see, and with how "fast" google changes service limits like that the only way that somewhat future proofs it by not relying on more gscript functions; i don't think they will ever touch or remove the baseline functions of the file I/O subsystem.

it's definitely a PITA to setup but it covers everything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants