proposal: runtime: memory awareness exposure in runtime like Objective-C didReceiveMemoryWarning

[Allendar]

For quite some years I've build iOS applications and absolutely loved the https://developer.apple.com/documentation/uikit/uiviewcontroller/1621409-didreceivememorywarning?preferredLanguage=occ functionality in Objective-C. One of my bigger enterprise app used huge chunks of caches to make the app run smooth on every front. To make sure the app could use it's maximum amount of cache at all times without violating iOS's policies I used didReceiveMemoryWarning a lot to trim some fat off the caches whenever the OS ran low on RAM. This way there was no need to pre-define cache-sizes per se.

Now I'm building a soon-to-be released open-source web-framework, which uses many tiny caches, but could potentially get millions of fills. Instead of predetermening cache sizes it would be amazing if you could bind a listener say like runtime.didReceiveMemoryWarning(func callbackFunc(context)), where context would even hold info like how much RAM is still left or that you could even trigger it at per certain time-intervals when bound and at which tolerance (say 70% OS RAM used).

This functionality would make it much easier to give flexibility to web-apps to grow their caches dynamically alongside other processes running like a database, mailing and queueing services and easily cleanup whenever needed. This way a web-app can scale intensely varying from small servers to enterprise-grade services, without requiring any extra setup.

I'm not very good at explaining, so please ask away if my request seems unclear. Maybe there's already an existing library or feature for this that I'm not aware of. If so; please let me know and render this feature request invalid.

[ianlancetaylor]

@aclements has a plan for this but I couldn't find an open issue for it.

[Allendar]

I'm curious if Linux/macOS/Windows have native signals that can be listened for or subscribed to for these kind of events.

[aclements]

Hi @Allendar. Are you asking for support for a specific OS API (though few OSes have a concept of a memory warning like iOS does), or something like CL 46751 (issue #16843)? The latter doesn't itself use any OS support but does give feedback on how much memory a Go process is using.

[Allendar]

Hi @aclements. Thank you for your quick response! The amount of memory the process itself uses is not really relevant in this case. It's really about the concept that the process can be aware of the fact that the system is running out of it's total memory, so it can run it's own internal cleanups. This would not only be useful for web-apps, but also for any app on any OS in general.

In my case the target will probably be Linux for webservers, but I develop on macOS and have some people help me from time-to-time on Windows. Even tho production deployments will probably always be Linux.

Doing a quick online search I can't really find much details on OS' their behaviour about this. This would be a great addition tho. Maybe we really have to dig into OS-specific manuals to find out what the possibilities are for this.

[Allendar]

It also states in the iOS manual (https://developer.apple.com/documentation/uikit/uiviewcontroller/1621409-didreceivememorywarning?preferredLanguage=occ) that you can only subscribe to the event but not trigger the listener yourself (which is kind of obvious in this case I guess).

[Allendar]

Having a GC-like cycler running that could access RAM stats per OS would probably be almost enough to do a threshold check on bytes still free in OS memory.

[aclements]

Doing a quick online search I can't really find much details on OS' their behaviour about this. This would be a great addition tho. Maybe we really have to dig into OS-specific manuals to find out what the possibilities are for this.

I agree it would be great if OSes generally supported this, but unfortunately they don't. Linux certainly doesn't have an API for this and I'm not aware of such APIs for macOS or Windows (though I'm less familiar with their OS APIs).

An application can query the OS for how much free memory there is and drop caches if this gets low, but this doesn't require any Go runtime support. This can get messy if multiple applications are doing this at the same time (what if they have different thresholds? what if they query at different times?), but might suffice in your situation.

[Allendar]

That makes sense. I don't think high velocity isn't very important on long running applications per se. The iOS concept idea is mainly that the OS wouldn't be forced to suddenly kill sleeping apps. The concept is pretty cool, but could probably cause unnecessary simultaneous cleanups or (counterproductive) cleanup-loops if another process is constantly hogging memory over and over.

Do you have an example to query a Linux kernel for these kind of statistics? Maybe I can find one for macOS. If I could gather a solid mechanic for all 3 big OS' it might be worth for a package or maybe even an internals addition.

I was just doodling that it could look something like this (but probably not with runtime):

runtime.didReceiveMemoryWarning(5 * time.Second, func (ctx context) {
	if ctx.PercentageUsed > 70 {
		// Do cleanup
	}
	// OR
	if ctx.MemoryBytesFree < 250<<20 {
		// Do cleanup
	}
})

[aclements]

Linux gives quite a bit of information about memory usage in /proc/meminfo. Note, however, that "free memory" is a lot more complicated than it sounds. The OS will try to use up any available memory with caches itself. What's the value of the OS's caches relative to your caches? For example, if your process's caches grow to the point where the OS starts evicting its cache of your process's code pages, things will get very slow, even though there's technically still memory available.

[Allendar]

In the current state if I would deploy a web app on an instance with 512MB RAM for a CMS with say 2000 visitors per hour. If all caches stay within bounds, all is well. But if I don't manually restrict them they could possibly cap up the full extend of the memory.

Say the visitors per hour go up to a 10000 and I constantly serve more cached database blog-entries, pages etc.. If I would upgrade my server-instance to 2GB I would need to reconfigure my cache limits to to an estimate, also still not being sure what the growth of the app's data might have on other running processes like database/queues/etc.

Instead of having the new framework's users having to manually configure these caps I'm trying to find ways to make these kind of configurations less painful and deployment easier even for less technical people. Or maybe say having less error-prone setups.

I could detect max RAM on startup and dynamically set the caps on some guess, but I'm not sure what are safe boundaries. That's kind of why the didReceiveMemoryWarning principal came to mind.

[aclements]

Sure, that all makes sense. I'm just saying that what Linux provides is /proc/meminfo and that, in general, this is a hard problem. :)

[Allendar]

Thanks for the feedback 👍. I will look into more the coming weekend. If I find more useful information I will surely provide it here :)

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)