Exception generating thumbnail image: Insufficient memory (case 4)

[rbillimo]

Expected behavior

When generating multiple thumbnails for images of size 8 MB and dimension 4000 x 6016px I expect the thumbnails for these to generate correctly.

Actual behavior

However instead of the Thumbnails generating correctly I am getting the following error

2017-03-29 16:33:22,089 [http-0.0.0.0-8080-16] DEBUG thumbnail.CustomThumbnailController:108 - Exception generating thumbnail image: Insufficient memory (case 4)
javax.imageio.IIOException: Insufficient memory (case 4)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.readImage(Native Method)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.readInternal(JPEGImageReader.java:1100)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.read(JPEGImageReader.java:915)
at net.coobird.thumbnailator.tasks.io.InputStreamImageSource.read(Unknown Source)
at net.coobird.thumbnailator.tasks.SourceSinkThumbnailTask.read(Unknown Source)
at net.coobird.thumbnailator.Thumbnailator.createThumbnail(Unknown Source)
at net.coobird.thumbnailator.Thumbnails$Builder.toFile(Unknown Source)
at thumbnail.CustomThumbnailController.generateThumbnailImage(CustomThumbnailController.java:138)
at thumbnail.CustomThumbnailController.handleRequestInternal(CustomThumbnailController.java:79)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.interwoven.ui.core.util.UtilityFilter.doFilter(UtilityFilter.java:85)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.interwoven.ui.core.auth.AuthenticationFilter.doFilter(AuthenticationFilter.java:205)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:96)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.autonomy.shannon.filter.CustomSecurityFilter.doFilter(CustomSecurityFilter.java:120)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:584)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
at java.lang.Thread.run(Thread.java:619)

Steps to reproduce the behavior

The following code is being invoked to generate a thumbnail. If you invoke this code for 4 images of the size i have mentioned the problem should be reproducible.

    Thumbnails.of(new InputStream[] { new BufferedInputStream(((CSSimpleFile)file).getBufferedInputStream(false)) }).size(getWidth(), getHeight()).outputFormat(replacedExtension ? "png" : file.getVPath().getExtension()).useExifOrientation(false).toFile(thumbnailFile);

Environment

• OS vendor and version: Windows 2003
• JDK vendor and version: JDK 1.6.0_12
• Thumbnailator version: 0.3.10 (Version being used) & 0.4.8 (Also reproduced on latest version)

[coobird]

Thank you for filing this issue.

I just want to double-check one thing. You mention that the image dimensions are "4000 x 60016px" -- that's a 240 megapixel image, that seems like an unrealistically large image. Could you double-check the dimensions please? Also, what are the image formats of the input?

That said, handling large images leading to OutOfMemoryErrors are a well-known issue (such as #105, #62, and in fact #1 as well), and in the current state of Thumbnailator, large input images will require more heap for processing, as it first opens up the image on the JVM heap.

Just as a reference, could you also mention what the heap size is on your JVM where you've experienced this problem?

Finally, there was one workaround added in 0.4.8 (#69) which attempts to calculate if a OutOfMemoryError could occur, and attempt to avoid it. (You can read more about it in the Changes for 0.4.8.) The workaround can be invoked by adding the -Dthumbnailator.conserveMemoryWorkaround=true JVM option. Just as a caution, this might degrade the final result somewhat, and it's a workaround that isn't guaranteed to stick around forever.

[rbillimo]

Hi

Thank you for your response. I made a initial typo the size of the image is 4000 x 6016. The image format is jpg. I am using the following JVM settings

-Xss256k -Xms775m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=256m -Dthumbnailator.conserveMemoryWorkaround=true on version 0.4.8

I have reviewed the other well known issues and i believe this issue is a little different. In particular the problem does not happen if one thumbnail is generated at a time. The problem happens when mutliple thumbnails are generated in parallel. I have created the code below that should enable you to reproduce the issue.

	public static class MyRunnable implements Runnable {
	private final String inputFileName, thumbnailFileName;

	MyRunnable(String inputFileName, String thumbnailFileName) {
		this.inputFileName = inputFileName;
		this.thumbnailFileName = thumbnailFileName;

	}

	@Override
	public void run() {
		try{
			FileInputStream  file = new FileInputStream (inputFileName);
			Thumbnails.of(new InputStream[] { new BufferedInputStream((file)) }).size(253, 380).outputFormat("jpg").useExifOrientation(false).toFile(thumbnailFileName);  
		}catch (Exception e){
			e.printStackTrace();
		}
	}
}


public static void main(String args []) throws IOException{
	ExecutorService executor = Executors.newFixedThreadPool(6);
	Runnable worker = new MyRunnable("C:\\temp\\1.jpg", "C:\\temp\\thumb\\1.jpg");
	Runnable worker2 = new MyRunnable("C:\\temp\\2.jpg", "C:\\temp\\thumb\\2.jpg");
	Runnable worker3 = new MyRunnable("C:\\temp\\3.jpg", "C:\\temp\\thumb\\3.jpg");
	Runnable worker4 = new MyRunnable("C:\\temp\\4jpg", "C:\\temp\\thumb\\4.jpg");
	Runnable worker5= new MyRunnable("C:\\temp\\5.jpg", "C:\\temp\\thumb\\5.jpg");
	Runnable worker6 = new MyRunnable("C:\\temp\\6.jpg", "C:\\temp\\thumb\\6.jpg");
	Runnable worker7 = new MyRunnable("C:\\temp\\7.jpg", "C:\\temp\\thumb\\7.jpg");
	Runnable worker8 = new MyRunnable("C:\\temp\\8.jpg", "C:\\temp\\thumb\\8.jpg");
	Runnable worker9= new MyRunnable("C:\\temp\\9.jpg", "C:\\temp\\thumb\\9.jpg");
	Runnable worker10 = new MyRunnable("C:\\temp\\10.jpg", "C:\\temp\\thumb\\10.jpg");
	Runnable worker11 = new MyRunnable("C:\\temp\\11.jpg", "C:\\temp\\thumb\\11.jpg");
	Runnable worker12 = new MyRunnable("C:\\temp\\12.jpg", "C:\\temp\\thumb\\12.jpg");

	executor.execute(worker);
	executor.execute(worker2);
	executor.execute(worker3);
	executor.execute(worker4);
	executor.execute(worker5);
	executor.execute(worker6);
	executor.execute(worker7);
	executor.execute(worker8);
	executor.execute(worker9);
	executor.execute(worker10);	
	executor.execute(worker11);
	executor.execute(worker12);
	
	executor.shutdown();
	// Wait until all threads are finish
	while (!executor.isTerminated()) {

	}
	System.out.println("\nFinished all threads");
}

One observation is that the issue is more easily reproducible on Java 6 than it is on Java 8. The code above will result in the error i reported on Java 6 but not on Java 8.

[coobird]

Thank you for adding more details, all of which are very relevant.

First of all, let me discuss how you can come up with a ballpark estimate of how much heap memory that's required -- internally (when the memory conservation workaround is not enabled), the peak memory usage will be at least the # pixels times 4 bytes. In this particular case, that would be 4000 x 6016 x 4 byte = more than 96 MB, and possibly the total memory used could be over double that amount. (That's because the original image, intermediate image, along with memory used by the JPEG decoder and all other overhead exists.)

For the sake of discussion, let's say 150 MB of memory is used at the peak of each image resize. Since you present a case of using a thread pool containing 6 threads, so in the worst case scenario, the JVM could be resizing 6 images at the same time, which would require 900 MB of memory right there. Even a heap of 1 GB could potentially overflow in this situation.

If the conserveMemoryWorkaround is being used however, the amount of memory used would be quite a bit less. I can't pinpoint the amount, but the memory size used for the "original" image is reduced to a maximum of 1200 x 1200 (if I recall correctly) which is about 6 MB. As always there's overhead, but it would be quite a bit smaller than 150 MB.

One thing that I just noticed right now is that the exception is not a OutOfMemoryError but an exception thrown by the JPEG decoder that comes with Java. I was assuming it was an OutOfMemoryError but it is not.

Since I was curious, I searched for "Insufficient memory (case 4)" and found that it shows up in non-Java libraries (and operating systems!) as well, and guessed (correctly) that it comes from libjpeg. And that's caused by a malloc failing here. malloc failing is a situation that's not seen often, but is an indication that the OS is getting to the point it's having a hard time allocating new memory. If you're really curious, here's an interesting Stack Overflow question about it.

Summing it up, I'm going to guess that the situation is that the JVM has enough memory allocated to it by the OS, but the JPEG decoder which is (probably) calling native code of libjpeg, and libjpeg is unable to obtain more free memory from the OS. You may want to look at how much (virtual) memory that the OS has free at the time you encounter this error. If my guess is correct, it would mean that you'll need to reduce the amount of parallel processing, and adding more heap (-Xmx) to the JVM probably isn't going to alleviate the situation.

I really have no answer about Java 8 being less prone to this issue than Java 6, aside from Java 8 probably has an improved garbage collector than Java 6, but if my guess from the previous paragraph is true, it shouldn't make a big difference.

What can be said about this issue from the standpoint of Thumbnailator? -- not a whole lot than what has been discussed in Issue #1 -- there needs to be a better way to handle large images.

For your particular scenario, you may want to re-evaluate whether your environment can handle multiple concurrent processing. Maybe reducing the thread pool size may reduce the occurrences of this issue.

[rbillimo]

Hi

Thank you for your update. I believe it has been very useful. I retested with a 32 bit version of Java 8 and have reproduced the issue on that. If i monitor in task manager i see that during the execution of my program it is reaching the memory limit of a 32 bit process and hence the reason i am getting the errors. With a 64 bit version of Java the problem is not happening and i see the Java process growing to around 1.8 GB in size.

If i understand your update correctly as the issue is happening from native code of libjpeg there is actually nothing that can be done to stop the issue on a 32 bit version of Java. (excluding changing the implementation of the program to have less concurrent processing).

Thank you again for your time and inputs.

[coobird]

I'm glad you were able to investigate deeper into the issue.

If i understand your update correctly as the issue is happening from native code of libjpeg there is actually nothing that can be done to stop the issue on a 32 bit version of Java. (excluding changing the implementation of the program to have less concurrent processing).

Yes, basically, the more concurrent processing you have, the more peak memory capacity you will need. At least with the way Thumbnailator does processing, it keeps a copy of the full image in memory, so the peak memory is relatively large. This will continue to be the case until issue #1 is resolved.

[coobird]

Basically, to be addressed in Issue #1.