Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Better Scheduling/Load Balancing for activities #52

Merged
merged 58 commits into from
Sep 1, 2021
Merged

Better Scheduling/Load Balancing for activities #52

merged 58 commits into from
Sep 1, 2021

Conversation

Romero027
Copy link
Collaborator

No description provided.

@ghost
Copy link

ghost commented Jun 16, 2021 •
edited by ghost
Loading

CLA assistant check
All CLA requirements met.

@sebastianburckhardt sebastianburckhardt mentioned this pull request Jun 17, 2021
Closed
sebastianburckhardt and others added 24 commits June 17, 2021 09:38
@sebastianburckhardt
keep partition prefix consistent with all the other tracing
a3e9a7e
@sebastianburckhardt
fix bug (failed to update estimated load when remote activities compl…
71fac85 
…ete)
@Romero027
Add aggresive scheduler mode
14d9cc8
@Romero027
fixed a typo
2db53f4
@Romero027
basic load monitor
8b22d5f
@Romero027
checkpoint
29499ad
@sebastianburckhardt
Merge branch 'main' into activity-scheduling
b0b5fca 
# Conflicts:
#	src/DurableTask.Netherite/TransportProviders/EventHubs/EventHubsConnections.cs
#	src/DurableTask.Netherite/TransportProviders/EventHubs/EventHubsProcessor.cs
@sebastianburckhardt
fix bug in ETW tracing
e349492
@sebastianburckhardt
change conditions for load monitor reporting
f261a5c
@sebastianburckhardt
use full tracing
b780b72
@sebastianburckhardt
trace more information for OffloadCommandReceived
94f2b4e
@sebastianburckhardt
change tracing of task messages to provide more consistent results
3add67b
@sebastianburckhardt
several package updates, including DurableTask.Core 2.5.6
1d909ce
@sebastianburckhardt
Merge branch 'pr/update-durabletask-2.5.6' into activity-scheduling
916ebfd
@sebastianburckhardt
Merge branch 'pr/fix-message-tracing' into activity-scheduling
b20427a
@sebastianburckhardt
update to account for tracing changes
26d581d
@Romero027
remove load monitor interval and add idle message
3d38f6d
@sebastianburckhardt
add latency field to ActivityCompleted and RemoteActivityResultReceived
badf584
@Romero027
added offload algorithms based on waiting time
d9e341f
@Romero027
fix overflow in filehash
fd4531e
@sebastianburckhardt
fixes to loadmonitor logic, autoscaler, and temporary info sender
5e7fafb
@sebastianburckhardt
Merge branch 'activity-scheduling' of https://github.com/microsoft/du…
4b21247 
…rabletask-netherite into activity-scheduling
@sebastianburckhardt
fix missing file and update scale tracing
4f151df
@sebastianburckhardt
change loadmonitor hosting so it moves less often
2c66ba0
sebastianburckhardt and others added 25 commits August 10, 2021 14:01
@sebastianburckhardt
fix so we don't pull more than mobile
078fe57
@sebastianburckhardt
fix dumb mistake
f4164fa
@sebastianburckhardt
fix formula
c3100c5
@sebastianburckhardt
need more conservative default estimate for completion time, otherwis…
533821e 
…e offload is too aggressive
@sebastianburckhardt
remove unnecessary constants.
cccd473
@sebastianburckhardt
add tracing for RTT
fe66e4d
@sebastianburckhardt
tweak parameters for RTT and smoothing
80916ac
@sebastianburckhardt
batchworker tracing for senders
76e1598
@sebastianburckhardt
use array instead of dictionary, fix concurrent modification exception
050ec69
@sebastianburckhardt
revise sender for load monitor events to send only latest state, and …
b8aa121 
…to do rate limiting
@sebastianburckhardt
fix tracing
a9b080a
@sebastianburckhardt
turn off all loadmonitor activity when the parameter ActivitySchedule…
46d6c1d 
…r is not set LoadMonitor
@sebastianburckhardt
implemented "Static" setting for ActivityScheduler
06b783f
@sebastianburckhardt
reduce severity of tracing in loadmonitor
079e69a
@Romero027
make filehash scale configurable
5e6ea6e
@sebastianburckhardt
Merge branch 'activity-scheduling' of https://github.com/microsoft/du…
ab914bf 
…rabletask-netherite into activity-scheduling
@sebastianburckhardt
update names, remove old algorithms
bad0649
@sebastianburckhardt
improve terminology and implement solicitation
bb833f2
@sebastianburckhardt
fixes and simplifications
fc728dd
@sebastianburckhardt
minor updates
5f6bd68
@sebastianburckhardt
simplify data structure for local activities
42a9884
@sebastianburckhardt
use smarter concurrency defaults, same as for default backend
5aaa8ae
@sebastianburckhardt
fix tracing in LoadMonitor
878fb09
@sebastianburckhardt
update filehash a bit to make it more uniform
8a80605
@sebastianburckhardt
add test series for comparing local vs locavore
1145551
@sebastianburckhardt sebastianburckhardt marked this pull request as ready for review August 25, 2021 15:26
@sebastianburckhardt
Merge branch 'main' into activity-scheduling
62512d0 
# Conflicts:
#	src/DurableTask.Netherite/Events/PartitionEvents/External/FromPartitions/ActivityOffloadReceived.cs
#	src/DurableTask.Netherite/OrchestrationService/Partition.cs
#	src/DurableTask.Netherite/StorageProviders/Faster/StoreWorker.cs
#	test/PerformanceTests/series/host.neth-loc.json
@sebastianburckhardt sebastianburckhardt changed the base branch from main to dev August 25, 2021 15:36
@sebastianburckhardt sebastianburckhardt merged commit 6b3b372 into dev Sep 1, 2021
@sebastianburckhardt sebastianburckhardt mentioned this pull request Jan 12, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Romero027 @sebastianburckhardt