Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor batch #20706

Closed
Closed

Conversation

christothes
Copy link
Member

@christothes christothes commented Apr 27, 2021

closes #20602

@ghost ghost added the Tables label Apr 27, 2021
@christothes christothes requested review from pakrym and jsquire April 27, 2021 19:53
@christothes christothes self-assigned this Apr 27, 2021
public virtual void UpdateEntity<T>(T entity, Azure.ETag ifMatch, Azure.Data.Tables.TableUpdateMode mode = Azure.Data.Tables.TableUpdateMode.Merge) where T : class, Azure.Data.Tables.ITableEntity, new() { }
public virtual void UpsertEntity<T>(T entity, Azure.Data.Tables.TableUpdateMode mode = Azure.Data.Tables.TableUpdateMode.Merge) where T : class, Azure.Data.Tables.ITableEntity, new() { }
public virtual void UpdateEntity(Azure.Data.Tables.ITableEntity entity, Azure.ETag ifMatch, Azure.Data.Tables.TableUpdateMode mode = Azure.Data.Tables.TableUpdateMode.Merge) { }
public virtual void UpsertEntity(Azure.Data.Tables.ITableEntity entity, Azure.Data.Tables.TableUpdateMode mode = Azure.Data.Tables.TableUpdateMode.Merge) { }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we choose when to provide only singular overload and when the IEnumerable one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's mostly an assumption that add is the common bulk scenario. We can always add more later if needed.

{
throw new InvalidOperationException(TableConstants.ExceptionMessages.BatchIsEmpty);
}
if (transactionalBatch._submitted)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to go against the batch being "just a model"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meant to remove that, thanks.

throw new Exception($"Unexpected error. Unable to {nameof(RemoveEntityOperation)}");
}
var itemsToRemove = new HashSet<string>(entities.Select(e => e.RowKey));
var newQueue = new ConcurrentQueue<string>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConcurrentQueue seems a bit heavyweight for what this method is doing.

/// <param name="entities"></param>
/// <returns></returns>
/// <exception cref="NotImplementedException"></exception>
public bool RemoveEntityOperations(IEnumerable<ITableEntity> entities)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would customers use the return value of this method?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If, for some reason, we failed to remove the operations they could try again or inspect the current entities in the batch and make a decision about why it failed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would the reason be if they passed a set of operations that a part of this batch?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps some are not, in fact, a part of the batch.
This is modeled after List.Remove and Dictionary.Remove

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But those don't take enumerables.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting it return void or that we return something more helpful, such as a list of the entities that could not be removed?

private Guid _changesetGuid;
internal readonly Dictionary<string, BatchItem> _requestLookup = new();
internal ConcurrentQueue<string> _requestMessages = new();
internal List<BatchItem> _submittedMessageList;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internal fields...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

@@ -32,12 +30,12 @@ internal TableBatchResponse(ConcurrentDictionary<string, (HttpMessage Message, R
/// <returns></returns>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Since this is a public member, should we consider adding content for <returns> or removing the tag?

internal readonly Dictionary<string, BatchItem> _requestLookup = new();
internal ConcurrentQueue<string> _requestMessages = new();
internal List<BatchItem> _submittedMessageList;
internal bool _submitted;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this may benefit from being volatile given its usage in the client when sending.

private Guid _changesetGuid;
internal readonly Dictionary<string, BatchItem> _requestLookup = new();
internal ConcurrentQueue<string> _requestMessages = new();
internal List<BatchItem> _submittedMessageList;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

{
throw new InvalidOperationException(TableConstants.ExceptionMessages.BatchCanOnlyBeSubmittedOnce);
}
transactionalBatch._submitted = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this operation fails, doesn't this prevent retries? Should we consider unsetting in the catch block, or is the intent to create a new batch and use that?

@@ -127,7 +127,7 @@ public async Task TablesTeardown()
/// <param name="partitionKeyValue">The partition key to create for the entity.</param>
/// <param name="count">The number of entities to create</param>
/// <returns></returns>
protected static List<TableEntity> CreateTableEntities(string partitionKeyValue, int count)
internal static List<TableEntity> CreateTableEntities(string partitionKeyValue, int count)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consider refactoring into a helper class if we're calling outside of when the base class is used?

@christothes
Copy link
Member Author

See #20723 for an alternate refactor.

@christothes
Copy link
Member Author

dupe of #20723

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tables Batch Arch board changes
3 participants