Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set index length limit for MySQL #14500

Merged
merged 4 commits into from
Oct 13, 2023
Merged

Set index length limit for MySQL #14500

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d1e60dc
Set index length limit for MySQL
MikeAlhayek Oct 12, 2023
e3b9342
Update migrations
MikeAlhayek Oct 12, 2023
27e175d
cleanup
MikeAlhayek Oct 12, 2023
5f7597d
undoing changes around Alias.
MikeAlhayek Oct 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions src/OrchardCore.Modules/OrchardCore.Alias/Migrations.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,19 @@ public int Create()
.Attachable()
.WithDescription("Provides a way to define custom aliases for content items."));

// NOTE: The Alias Length has been upgraded from 64 characters to 767.
// For existing SQL databases update the AliasPartIndex tables Alias column length manually.
// INFO: The Alias Length is now of 735 chars, but this is only used on a new installation.
Copy link
Member

@jtkech jtkech Oct 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said in #14491 (comment)

... DocumentId (26) is wrong as it is at most a bigint (size = 8 bytes = 2 chars size), the 2 booleans may not share the same bytes but one char size (4 bytes) for both is sufficient, anyway we kept a margin of 4 bytes.

Not so important to assume one "char" per boolean, but at least comment DocumentId (2) which here will allow Alias(738), and if we want to keep a margin of 4 bytes => Alias(734).

Then the following comments could be removed, and maybe revert back MaxAliasLength to its original value, from memory it was 1024, but maybe too much, I don't know, just for info.

OrchardCore/src/OrchardCore.Modules/OrchardCore.Alias/Models/AliasPart.cs

Lines 7 to 13 in f377a07

// Maximum length that MySql can support in an index under utf8mb4 collation is 768,
// minus 2 for the `DocumentId` integer (bigint size = 8 bytes = 2 character size),
// minus 26 for the `ContentItemId` and 1 for the 'Published' and 'Latest' bools.
// minus 4 to allow at least to add a new integer column.
public const int MaxAliasLength = 735;
public string Alias { get; set; }

Same remark for other indexes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jtkech thanks for pointing out the seize of DocumentId I know it was long but assumed it was a char :). Anyway, I update it.

SchemaBuilder.CreateMapIndexTable<AliasPartIndex>(table => table
.Column<string>("Alias", col => col.WithLength(AliasPart.MaxAliasLength))
.Column<string>("ContentItemId", c => c.WithLength(26))
.Column<bool>("Latest", c => c.WithDefault(false))
.Column<bool>("Published", c => c.WithDefault(true))
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + Alias (714) + ContentItemId (26) + Published (1) + Latest (1) = 768
SchemaBuilder.AlterIndexTable<AliasPartIndex>(table => table
.CreateIndex("IDX_AliasPartIndex_DocumentId",
"DocumentId",
"Alias",
"Alias(714)",
"ContentItemId",
"Published",
"Latest")
Expand Down
4 changes: 0 additions & 4 deletions src/OrchardCore.Modules/OrchardCore.Alias/Models/AliasPart.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,6 @@ namespace OrchardCore.Alias.Models
{
public class AliasPart : ContentPart
{
// Maximum length that MySql can support in an index under utf8mb4 collation is 768,
// minus 2 for the `DocumentId` integer (bigint size = 8 bytes = 2 character size),
// minus 26 for the `ContentItemId` and 1 for the 'Published' and 'Latest' bools.
// minus 4 to allow at least to add a new integer column.
public const int MaxAliasLength = 735;

public string Alias { get; set; }
Expand Down
141 changes: 81 additions & 60 deletions src/OrchardCore.Modules/OrchardCore.ContentFields/Indexing/SQL/Migrations.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<TextFieldIndex>(table => table
.CreateIndex("IDX_TextFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -80,12 +82,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Boolean (1) + Published (1) + Latest (1) = 767 (less than 768)
SchemaBuilder.AlterIndexTable<BooleanFieldIndex>(table => table
.CreateIndex("IDX_BooleanFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Boolean",
"Published",
"Latest")
Expand All @@ -111,12 +115,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<NumericFieldIndex>(table => table
.CreateIndex("IDX_NumericFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -149,12 +155,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<DateTimeFieldIndex>(table => table
.CreateIndex("IDX_DateTimeFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -187,12 +195,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<DateFieldIndex>(table => table
.CreateIndex("IDX_DateFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -226,12 +236,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<ContentPickerFieldIndex>(table => table
.CreateIndex("IDX_ContentPickerFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -264,12 +276,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<TimeFieldIndex>(table => table
.CreateIndex("IDX_TimeFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -309,12 +323,15 @@ public int Create()
"Latest")
);


// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<LinkFieldIndex>(table => table
.CreateIndex("IDX_LinkFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -355,12 +372,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<HtmlFieldIndex>(table => table
.CreateIndex("IDX_HtmlFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand All @@ -386,12 +405,14 @@ public int Create()
"Latest")
);

// The index in MySQL can accommodate up to 768 characters or 3072 bytes.
// DocumentId (26) + ContentType (246) + ContentPart (246) + ContentField (246) + Published (1) + Latest (1) = 766 (less than 768)
SchemaBuilder.AlterIndexTable<MultiTextFieldIndex>(table => table
.CreateIndex("IDX_MultiTextFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -453,9 +474,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<TextFieldIndex>(table => table
.CreateIndex("IDX_TextFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand All @@ -481,9 +502,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<BooleanFieldIndex>(table => table
.CreateIndex("IDX_BooleanFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Boolean",
"Published",
"Latest")
Expand All @@ -501,9 +522,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<NumericFieldIndex>(table => table
.CreateIndex("IDX_NumericFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand All @@ -528,9 +549,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<DateTimeFieldIndex>(table => table
.CreateIndex("IDX_DateTimeFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand All @@ -555,9 +576,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<DateFieldIndex>(table => table
.CreateIndex("IDX_DateFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand All @@ -582,9 +603,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<ContentPickerFieldIndex>(table => table
.CreateIndex("IDX_ContentPickerFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand All @@ -609,9 +630,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<TimeFieldIndex>(table => table
.CreateIndex("IDX_TimeFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand All @@ -636,9 +657,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<LinkFieldIndex>(table => table
.CreateIndex("IDX_LinkFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down Expand Up @@ -673,9 +694,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<HtmlFieldIndex>(table => table
.CreateIndex("IDX_HtmlFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand All @@ -692,9 +713,9 @@ public int UpdateFrom3()
SchemaBuilder.AlterIndexTable<MultiTextFieldIndex>(table => table
.CreateIndex("IDX_MultiTextFieldIndex_DocumentId_ContentType",
"DocumentId",
"ContentType",
"ContentPart",
"ContentField",
"ContentType(246)",
"ContentPart(246)",
"ContentField(246)",
"Published",
"Latest")
);
Expand Down