Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace middleware with simple endpoints #17373

Merged
merged 49 commits into from
Feb 7, 2025
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
df10660
Use simple endpoints for liquid intellisense script
jbytes1027 Jan 17, 2025
217bf0e
Add caching to get intellisense script
jbytes1027 Jan 17, 2025
6302024
Improve GetHashCode function
jbytes1027 Jan 17, 2025
081cf78
Disable warning for line
jbytes1027 Jan 17, 2025
ab8db2a
Use simple endpoints for facebook script endpoints
jbytes1027 Jan 17, 2025
8777c2f
Inject shell configuration
jbytes1027 Jan 21, 2025
85ec062
Update cachekey
jbytes1027 Jan 21, 2025
47e3187
Use unique urls instead of cache validation for fbsdk.js
jbytes1027 Jan 21, 2025
ce1f203
Fix formatting
jbytes1027 Jan 21, 2025
432cb1d
Replace GetHashCode with xxHash
jbytes1027 Jan 22, 2025
2a83296
Encode parameters
jbytes1027 Jan 22, 2025
d3204ca
Add culture inheritance to resource dependencies
jbytes1027 Jan 24, 2025
fe193a6
Rename fb.js to init.js
jbytes1027 Jan 24, 2025
1be4ffa
Add culture to fb script registrations
jbytes1027 Jan 24, 2025
3f099ac
Refactor fetch sdk endpoint
jbytes1027 Jan 24, 2025
854bec7
Fix typo
jbytes1027 Jan 24, 2025
e596b25
Disable caching facebook init.js
jbytes1027 Jan 25, 2025
2160d1b
Disable caching liquid-intellisense.js
jbytes1027 Jan 25, 2025
773538c
Merge branch 'main' into Fix15629
jbytes1027 Jan 28, 2025
58324ec
Improvements
sebastienros Jan 30, 2025
d5b10c9
Fix shared script cache key
jbytes1027 Jan 30, 2025
8e1de2c
Fix endpoint mapping
jbytes1027 Jan 30, 2025
48da77c
Ensure correct hash
jbytes1027 Jan 30, 2025
274c530
Cleanup structure and comments
jbytes1027 Jan 30, 2025
7c63037
Update cache timeout
jbytes1027 Jan 30, 2025
0d9a46e
Remove unused async
jbytes1027 Jan 30, 2025
3a64079
Add endpoint versions
jbytes1027 Jan 31, 2025
b27516e
Seperate facebook sdk endpoints into separate classes
jbytes1027 Jan 31, 2025
57fc7ca
Remove unused parameter
jbytes1027 Jan 31, 2025
b137ab4
Update cache key
jbytes1027 Jan 31, 2025
aa3c4ce
Use endpoint specific hashing
jbytes1027 Jan 31, 2025
54a9874
Ignore passed hash
jbytes1027 Jan 31, 2025
d3d2e0d
Fix cache key already used
jbytes1027 Jan 31, 2025
69b5100
Remove unused interpolation
jbytes1027 Jan 31, 2025
3b84be9
Fix outdated comment
jbytes1027 Jan 31, 2025
eaaaea0
Merge branch 'main' into sebros/james
jbytes1027 Jan 31, 2025
dfeea37
Disable caching liquid-intellisense.js
jbytes1027 Jan 25, 2025
5e322b8
Merge remote-tracking branch 'origin/Fix15629' into Fix15629
jbytes1027 Jan 31, 2025
e34fc45
Use InvariantCulture for ToString
jbytes1027 Jan 31, 2025
4250747
Missing using
sebastienros Jan 31, 2025
79f7e81
Updates
sebastienros Jan 31, 2025
203fe20
Correct failure response code
jbytes1027 Jan 31, 2025
f857e4a
Inline var
jbytes1027 Jan 31, 2025
54471f2
Fix formatting
jbytes1027 Jan 31, 2025
ecb1781
Add comments
jbytes1027 Jan 31, 2025
9936f87
Improve hash performance
jbytes1027 Jan 31, 2025
5fc5aeb
Move hash to version parameter
jbytes1027 Jan 31, 2025
65f0e1c
Refactor facebook script endpoints to use only one cache value
jbytes1027 Jan 31, 2025
00497b6
Fix non-existent response
jbytes1027 Jan 31, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
using System.Text;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Routing;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Primitives;
using OrchardCore.Environment.Shell.Configuration;
using OrchardCore.Facebook.Settings;
using OrchardCore.Settings;

#nullable enable

namespace OrchardCore.Facebook.Endpoints;

public static class GetSdkEndpoints
{
public static IEndpointRouteBuilder AddSdkEndpoints(this IEndpointRouteBuilder builder)
{
builder.MapGet("/OrchardCore.Facebook/sdk/fbsdk.js", HandleFbsdkScriptRequestAsync)
.AllowAnonymous()
.DisableAntiforgery();

builder.MapGet("/OrchardCore.Facebook/sdk/fb.js", HandleFbScriptRequestAsync)
.AllowAnonymous()
.DisableAntiforgery();

return builder;
}

private static uint GetHashCode(byte[] bytes)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This made me think some more about how we should handle etag here. What will happen is that when the cache is stale on the client, it will send a request with the etag it has, which represents the version of the content it has for this resource (a hash can be considered a version indifferently in this case). And in the case of FB scripts (and others) they will not vary ever, until we actually ship another version of Orchard. So we can use a constant value for the etag, that will get a new value when we change the script. This etag could be done manually, or initialized based on the content on first use.

I am not sure about culture specific requests though, I haven't checked if the script depends on the requested culture, or the current OC's culture.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also play with the url of the script to have different ones per version and culture. This way the cache would be infinite. And no etag to maintain. We could still cache the content to serve the same to all clients.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the dotnet component that does it already: https://learn.microsoft.com/en-us/dotnet/api/system.hashcode.addbytes?view=net-9.0

Would be risky as the language codes can differ by two chars. E.g. "us" and "su" add to the same value. Thus the multiplication hash *= 17 is needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1- The hashcode algorithm in dotnet is probably better than yours (euphemism), and faster. You should really trust it. And if you don't: https://sharplab.io/#v2:C4LgTgrgdgNAJiA1AHwAICYAMBYAUBgRj1UwAJUCA6AFQFMAPYAbjzwDcBDMUgCwNIC8pKLQDupABIcAzjwDCAezi0AFAEoWuPpQCCcOACEAnsFrSVAUSgBjJQEsoAc0oBVagDEAHJQDitYMam5gBEPLQANuEKALQQ0sFqGqy4nNw86ILCYpIy8kqqSVrouvqBZpY29k6uHt5+ASbloRFR0dIQCYXEBACcKtrUClKyisrqhRR96TRDuaMFGkA===
2- Since the script are actually static per deployment we can make up a static key for the etag not based on a hash, but something like 1.us (version/culture).
3- We don't need an etag since we can cache forever if the url has the version and culture in it (can be arguments in the querystring ?v=1). Keeping the vary-by header based on accept-language should tell the client to cache independently.

If it's getting too complicated feel free to request a pair programming session.

This comment was marked as outdated.

Copy link
Contributor Author

@jbytes1027 jbytes1027 Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hashcode algorithm in dotnet is probably better

Yes but HashCode purposely won't work across process. There is System.IO.Hashing for non-cryptographic hashing if you want to introduce a dependency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to this endpoint to use infinite caching and versioning instead of etag and cache validation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HashCode purposely won't work across process

Caching won't either. So technically not required to have a stable hashcode. It's stable as long as the process lives. Otherwise use xxHash. (in System.IO.Hashing)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implimented xxHash hashing. This enables cache revalidation to work across process/restarts thanks to these lines:

        // Can be true if the cache was reset after the last client request
        if (requestETag.Equals(eTag))
        {
            return Results.StatusCode(304);
        }

{
uint hash = 0;
foreach (byte b in bytes)
{
hash += b;
hash *= 17;
}
return hash;
}

private static IResult HandleFbsdkScriptRequestAsync([FromQuery(Name = "lang")] string language, [FromQuery(Name = "sdkf")] string sdkFilename, HttpContext context, IMemoryCache cache)
{
// Set the cache timeout to the maximum allowed length of one year
// max-age is needed because immutable is not widly supported
context.Response.Headers.CacheControl = $"public, max-age=31536000, immutable";

string scriptCacheKey = $"~/OrchardCore.Facebook/sdk/fbsdk.js?sdkf={sdkFilename}&lang={language}";

var scriptBytes = cache.Get(scriptCacheKey) as byte[];
if (scriptBytes == null)
{
// Note: Update script version in ResourceManagementOptionsConfiguration.cs after editing
scriptBytes = Encoding.UTF8.GetBytes($@"(function(d){{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty sure it could use the same "preamble/end" caching mechanism I showed today. And then nothing needs to use memory cache. Unless we decide to return the result in a single byte[].

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this endpoint is there any downside to using the IMemoryCache? The beginning and end would still be cached in memory of course, just without IMemoryCache. Also splitting it up makes it less readable.

The other two endpoints get the advantage of being able to hash the entire byte array and thus revalidation.

var js, id = 'facebook-jssdk'; if (d.getElementById(id)) {{ return; }}
js = d.createElement('script'); js.id = id; js.async = true;
js.src = ""https://connect.facebook.net/{language.Replace('-', '_')}/{sdkFilename}"";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not safe (encoding).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed again: Refactored to use dependency injection.

d.getElementsByTagName('head')[0].appendChild(js);
}} (document));");

cache.Set(scriptCacheKey, scriptBytes);
}

return Results.Bytes(scriptBytes, "application/javascript");
}

private static async Task<IResult> HandleFbScriptRequestAsync(HttpContext context, ISiteService siteService, IMemoryCache cache, IShellConfiguration shellConfiguration)
{
var settings = await siteService.GetSettingsAsync<FacebookSettings>();

if (string.IsNullOrWhiteSpace(settings.AppId))
{
return Results.Forbid();
}

context.Response.Headers.CacheControl = shellConfiguration.GetValue(
"StaticFileOptions:CacheControl",
// Fallback value
$"public, max-age={TimeSpan.FromDays(30).TotalSeconds}");

// Assumes IfNoneMatch has only one ETag for performance
var requestETag = context.Request.Headers.IfNoneMatch;
if (!StringValues.IsNullOrEmpty(requestETag) && cache.Get(requestETag) != null)
{
context.Response.Headers.ETag = requestETag;

return Results.StatusCode(304);
}

string scriptCacheKey = $"/OrchardCore.Facebook/sdk/fb.js.{settings.AppId}.{settings.Version}";

var scriptBytes = (byte[]?)cache.Get(scriptCacheKey);
if (scriptBytes == null)
{
// Generate script
var options = $"{{ appId:'{settings.AppId}',version:'{settings.Version}'";
options = string.IsNullOrWhiteSpace(settings.FBInitParams)
? string.Concat(options, "}")
: string.Concat(options, ",", settings.FBInitParams, "}");
scriptBytes = Encoding.UTF8.GetBytes($"window.fbAsyncInit = function(){{ FB.init({options});}};");

cache.Set(scriptCacheKey, scriptBytes);
}

// False positive: No comparison is taking place here
#pragma warning disable RS1024
// Uses a custom GetHashCode because Object.GetHashCode differs across processes
StringValues eTag = $"\"{GetHashCode(scriptBytes)}\"";
#pragma warning restore RS1024

// Mark that the eTag corresponds to a fresh file
cache.Set(eTag, true);

context.Response.Headers.ETag = eTag;

// Can be true if the cache was reset after the last client request
if (requestETag.Equals(eTag))
{
return Results.StatusCode(304);
}

return Results.Bytes(scriptBytes, "application/javascript");
}
}
Original file line number Diff line number Diff line change
@@ -1,25 +1,38 @@
using System.Globalization;
using Microsoft.Extensions.Options;
using OrchardCore.Facebook.Settings;
using OrchardCore.ResourceManagement;
using OrchardCore.Settings;

namespace OrchardCore.Facebook;

public sealed class ResourceManagementOptionsConfiguration : IConfigureOptions<ResourceManagementOptions>
{
private static readonly ResourceManifest _manifest;
private readonly ResourceManifest _manifest;
private readonly ISiteService _siteService;

static ResourceManagementOptionsConfiguration()
public ResourceManagementOptionsConfiguration(ISiteService siteService)
{
_siteService = siteService;

_manifest = new ResourceManifest();

_manifest
.DefineScript("fb")
.SetDependencies("fbsdk")
.SetUrl("~/OrchardCore.Facebook/sdk/fb.js");
}

public async void Configure(ResourceManagementOptions options)
{
var settings = await _siteService.GetSettingsAsync<FacebookSettings>();
var language = CultureInfo.CurrentUICulture.Name;

_manifest
.DefineScript("fbsdk")
.SetUrl("~/OrchardCore.Facebook/sdk/fbsdk.js");
}
// v parameter is for cache busting
.SetUrl($"~/OrchardCore.Facebook/sdk/fbsdk.js?sdkf={settings.SdkJs}&lang={language}&v=1.0");

public void Configure(ResourceManagementOptions options) => options.ResourceManifests.Add(_manifest);
options.ResourceManifests.Add(_manifest);
}
}
64 changes: 0 additions & 64 deletions src/OrchardCore.Modules/OrchardCore.Facebook/ScriptsMiddleware.cs

This file was deleted.

6 changes: 5 additions & 1 deletion src/OrchardCore.Modules/OrchardCore.Facebook/Startup.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using OrchardCore.DisplayManagement.Handlers;
using OrchardCore.Facebook.Endpoints;
using OrchardCore.Environment.Shell.Configuration;
using OrchardCore.Facebook.Drivers;
using OrchardCore.Facebook.Filters;
using OrchardCore.Facebook.Recipes;
Expand All @@ -20,11 +22,13 @@ public sealed class Startup : StartupBase
{
public override void Configure(IApplicationBuilder builder, IEndpointRouteBuilder routes, IServiceProvider serviceProvider)
{
builder.UseMiddleware<ScriptsMiddleware>();
routes.AddSdkEndpoints();
}

public override void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<IShellConfiguration, ShellConfiguration>();

services.AddPermissionProvider<Permissions>();
services.AddNavigationProvider<AdminMenu>();

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
using System.Text;
using Fluid;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.Primitives;
using Microsoft.Net.Http.Headers;
using OrchardCore.DisplayManagement.Liquid;
using OrchardCore.Environment.Shell.Configuration;

#nullable enable

namespace OrchardCore.Liquid.Endpoints.Scripts;

public static class GetIntellisenseEndpoint
{
public static IEndpointRouteBuilder AddGetIntellisenseScriptEndpoint(this IEndpointRouteBuilder builder)
{
builder.MapGet("OrchardCore.Liquid/Scripts/liquid-intellisense.js", HandleRequest)
.AllowAnonymous()
.DisableAntiforgery();

return builder;
}

private static byte[] GenerateScript(IServiceProvider serviceProvider)
{
var templateOptions = serviceProvider.GetRequiredService<IOptions<TemplateOptions>>();
var liquidViewParser = serviceProvider.GetRequiredService<LiquidViewParser>();
var filters = string.Join(',', templateOptions.Value.Filters.Select(x => $"'{x.Key}'"));
var tags = string.Join(',', liquidViewParser.RegisteredTags.Select(x => $"'{x.Key}'"));
var script = $@"[{filters}].forEach(value=>{{if(!liquidFilters.includes(value)){{ liquidFilters.push(value);}}}});
[{tags}].forEach(value=>{{if(!liquidTags.includes(value)){{ liquidTags.push(value);}}}});";

return Encoding.UTF8.GetBytes(script);
}

private static uint GetHashCode(byte[] bytes)
{
uint hash = 0;
foreach (byte b in bytes)
{
hash += b;
hash *= 17;
}
return hash;
}

private static IResult HandleRequest(HttpContext context, IMemoryCache cache, IShellConfiguration shellConfiguration)
{
context.Response.Headers.CacheControl = shellConfiguration.GetValue(
"StaticFileOptions:CacheControl",
// Fallback value
$"public, max-age={TimeSpan.FromDays(30).TotalSeconds}");

// Assumes IfNoneMatch has only one ETag for performance
var requestETag = context.Request.Headers.IfNoneMatch;
if (!StringValues.IsNullOrEmpty(requestETag) && cache.Get(requestETag) != null)
{
context.Response.Headers.ETag = requestETag;

return Results.StatusCode(304);
}

const string scriptCacheKey = "OrchardCore.Liquid/Scripts/liquid-intellisense.js";

var scriptBytes = (byte[]?)cache.Get(scriptCacheKey);
if (scriptBytes == null)
{
scriptBytes = GenerateScript(context.RequestServices);

cache.Set(scriptCacheKey, scriptBytes);
}

// False positive: No comparison is taking place here
#pragma warning disable RS1024
// Uses a custom GetHashCode because Object.GetHashCode differs across processes
StringValues eTag = $"\"{GetHashCode(scriptBytes)}\"";
#pragma warning restore RS1024

// Mark that the eTag corresponds to a fresh file
cache.Set(eTag, true);

context.Response.Headers.ETag = eTag;

// Can be true if the cache was reset after the last client request
if (requestETag.Equals(eTag))
{
return Results.StatusCode(304);
}

return Results.Bytes(scriptBytes, "application/javascript");
}
}
Loading