Help
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Copilot Lvl 3
Message 1 of 9

Memory Error on my crop and stack program

I recently started learning programming and have finally written my first program. This program inputs all the images in a folder and crops them to a certain size. Then these images are then stacked vertically to produce a new image. So I have been running test images through my program and it appeared to work fine. But I have only been using 10 images at a time. When I decided to finally try the program for the purpose I intended, which requires over 500 images, the program crashes. I get the MemoryError. Since I just started, I wrote together whatever I thought would work, whether it was the most efficient method or not. If anyone has time, are there any blatant things in my code that just consume way too much resources. I have attached my first and second iteration of the program below.

from PIL import Image
from natsort import natsorted
import glob # Convert coordinate list into variables print("Type in the coordinates for the upper left (x1,y1) and bottom right (x2,y2) points")coordinates = list(map(int, input("Separate values with a space (x1 y1 x2 y2): ").strip().split()))[:4]x1, y1, x2, y2 = coordinatesprint("Generating image...") # creating lists to hold the initial files and the edited files
image_list = []
cropped_images = [] # Accessing all the files using the glob module # The function natsorted sorts values the way Windows does # Opens all the files with the variable img in folder and adds them one by one to a list named image_list for filename in natsorted(glob.glob("C:\\Users\\Alex\\Desktop\\One Dimensional Reaction Diffusion Program\\data\\*.tiff")):
img = Image.open(filename)
image_list.append(img) # for each image in the image_list # selected_region from previous user input # cropped_region applied the crop function for the images in the selected region # Cropping function
selected_region = (x1, y1, x2, y2)
cropped_region = image.crop(selected_region) # If cropped area is vertical, rotate into horizontal position if (y2 - y1) > (x2 - x1):
rotated_image = cropped_region.rotate(90, expand=1) # Expand 1 ensures image isn't cut off while rotating else:
rotated_image = cropped_region # Does nothing if image is horizontal
cropped_images.append(rotated_image) # appends all rotated_images to the list cropped_images # Size of each individual image
widths, heights = zip(*(i.size for i in cropped_images)) # Final image dimensions
max_width = max(widths)
total_height = sum(heights) # Create a new colored image (RGB)
new_img = Image.new('RGB', (max_width, total_height)) # Stacking function # to offset horizontally, need to change final image dimensions to sum(widths) and max(heights) # and change new_im.paste(img, (x_offset,0))
y_offset = 0 # Initial position is 0 for img in cropped_images: # Not sure if img will cause conflict because img used in above for loop
new_img.paste(img, (0, y_offset)) # (x,y) indicates which direction to offset in
y_offset += img.size[1] # location of new image is y_offset (which had position 0) + the height of the image
new_img
.save("C:\\Users\\Alex\\Desktop\\One Dimensional Reaction Diffusion Program\\output\\stacked.tiff", quality=95)

 

So, my original program was opening all the images into a list. Then they are cropped and saved to another list. I've been trying to write a single loop, but am unsure why my program won't run.

 

from PIL import Image
from natsort import natsorted
import glob

# Convert coordinate list into variables
print("Type in the coordinates for the upper left (x1,y1) and bottom right (x2,y2) points")
coordinates = list(map(int, input("Separate values with a space (x1 y1 x2 y2): ").strip().split()))[:4]
x1, y1, x2, y2 = coordinates
print("Generating image...")

# Accessing all the files using the glob module
# The function natsorted sorts values the way Windows does
# Opens all the files with the variable img in folder and adds them one by one to a list named image_list
for filename in natsorted(
glob.glob("C:\\Users\\Alex\\Desktop\\One Dimensional Reaction Diffusion Program\\data\\*.tiff")):
img = Image.open(filename)

# Cropping function
selected_region = (x1, y1, x2, y2) # selected region from previous user input
cropped_region = img.crop(selected_region) # taking opened image and cropping to selected region
if (y2 - y1) > (x2 - x1): # If cropped area is vertical, rotate into horizontal position
rotated_image = cropped_region.rotate(90, expand=1) # Expand 1 ensures image isn't cut off while rotating
else:
rotated_image = cropped_region # Does nothing if image is horizontal
img.close() # Closes the initially opened image

widths, heights = zip(*(i.size for i in rotated_image)) # Indicate sizes of rotated_image
max_width = max(widths) # Final image dimensions
total_height = sum(heights)
new_im = Image.new('RGB', (max_width, total_height)) # Create a new colored image (RGB)
y_offset = 0 # Initial position is 0
new_im.paste(rotated_image, (0, y_offset)) # (x,y) indicates which direction to offset in
y_offset += rotated_image.size[1] # location of new image is y_offset (which had position 0) + the height of the image

new_im.save("C:\\Users\\Alex\\Desktop\\One Dimensional Reaction Diffusion Program\\output\\stacked.tiff",
quality=95)

 

 

8 Replies
Highlighted
Commander Lvl 1
Message 2 of 9

Re: Memory Error on my crop and stack program

Hey! Do you mind posting the error message and the stack trace you are getting so I can try and help? 



*Mark helpful posts with Accept as Solution to help other users locate important info. Don't forget to give Kudos for great content!*
Highlighted
Copilot Lvl 3
Message 3 of 9

Re: Memory Error on my crop and stack program

For the first program I get:

Traceback (most recent call last):
File "C:/Users/Alex/PycharmProjects/pycharm/Combined Program.py", line 28, in <module>
cropped_region = image.crop(selected_region) # unsure if this is correct
File "C:\Users\Alex\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\Image.py", line 1105, in crop
self.load()
File "C:\Users\Alex\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\TiffImagePlugin.py", line 1071, in load
return super().load()
File "C:\Users\Alex\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\ImageFile.py", line 207, in load
self.load_prepare()
File "C:\Users\Alex\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\ImageFile.py", line 277, in load_prepare
self.im = Image.core.new(self.mode, self.size)
MemoryError

 

The second one I get:

 

Traceback (most recent call last):
File "C:/Users/Alex/PycharmProjects/pycharm/Combined ProgramV2.py", line 27, in <module>
widths, heights = zip(*(i.size for i in rotated_image)) # Indicate sizes of rotated_image
TypeError: 'Image' object is not iterable

Pilot Lvl 2
Message 4 of 9

Re: Memory Error on my crop and stack program

"MemoryError" basically means there wasn't enough RAM available for your program (see the Python documentation). Handling each image in turn instead of keeping them in RAM is a good solution for that.

 

Regarding this:

 

widths, heights = zip(*(i.size for i in rotated_image))

The problem seems to be that you use rotated_image as an iterable (e.g. a list), but its a single object instead. You'll have to adjust the size calculation to work with the loop, too, based on stated and size information for the current image. At a glance it seems you want the maximum width and height, so you could have variables that contain the maximum seen so far, and check for each image if you need to change the value.

 

Highlighted
Copilot Lvl 3
Message 5 of 9

Re: Memory Error on my crop and stack program

Thank you for the reply, I have removed the part with zip and added a new input to initially create the size of the final image. However, my images appears to keep stacking on top and rewriting each other. I have posted the code in a comment below.

Highlighted
Commander Lvl 2
Message 6 of 9

Re: Memory Error on my crop and stack program

@airtower-luna is correct, the MemoryError indicates python ran out of memory.

 

The first thing to try is in your original program, add these lines

img.Close()
gc.collect()

at the bottom of your "for filename" loop. [You'll probably have to add an "import gc" to the top of the program.] You're done with the original image after you crop it, so you need to tell Python that you're done. The gc.collect will hopefully force python to make that memory immediately available [python might wait a bit, so for a little while you're "using" extra memory you don't actually need].

 

You probably should add a counter and a print() statement to show you how many images you successfully load/crop before the program falls over. My gut tells me you'll not be able to load all 500 images into memory - at least for your target crop size.

 

The idea of a single pass of load/crop/append is a good one, but I would predict you'll run out of memory even earlier. That is because you have to allocate memory for the destination image first! The destination image, being (500 * cropHigh) * cropWide in size, takes up (nearly?) as much memory as your list of 500 * (cropHigh * cropWide) images. You failed to allocate the list, so you're not likely to allocate the destination.

 

I hope I'm wrong. You might have to use a smaller crop size or two sets of 250 images.

 

Are the images monochrome, greyscale or full-color? If either of the first, you could use a smaller image representation. Full-color requires 32 bits per pixel ...

Please follow-up to let us know how you made out. For good karma, mark a reply as the answer if it helped!

Highlighted
Copilot Lvl 3
Message 7 of 9

Re: Memory Error on my crop and stack program

First of all, thank you everyone for the advice.

Over the past day, I have edited my program to what it currently is. I now close the initial image after it has been cropped. I changed it to where I create the size of the final image before the loop begins. Then one image is opened, cropped, initial image is closed, and then the rotated image will paste on the image created outside the loop. However, now, instead of offsetting each paste, it pastes directly in the same position. I thought 

y_offset += rotated_image.size[1]

will increase the offset by n x rotated_image with each loop. I think if I can get this part to work, I think it will only ever have one image open at once, which can hopefully solve the memory issue. Regarding the gc, I'm not sure where that will go and I will add the print() statement in. Lastly, my images will be colored.

Updated program.

from PIL import Image
from natsort import natsorted
import glob

# Convert coordinate list into variables
print("Type in the coordinates for the upper left (x1,y1) and bottom right (x2,y2) points")
coordinates = list(map(int, input("Separate values with a space (x1 y1 x2 y2): ").strip().split()))[:4]
x1, y1, x2, y2 = coordinates
image_count = int(input("How many images are being stacked? "))
print("Generating image...")

# Generate final image first
# Width dependent on whether image is horizontal or vertical
if (y2 - y1) > (x2 - x1):
width = (y2 - y1)
height = (x2 - x1)
else:
width = (x2 - x1)
height = (y2 - y1)

# Width is constant, total_height accounts for height above and the total number of images
total_height = (image_count * height)
new_im = Image.new('RGB', (width, total_height)) # Create a new colored image (RGB)

# Accessing all the files using the glob module
# The function natsorted sorts values the way Windows does
# Opens all the files with the variable img in folder and adds them one by one to a list named image_list
for filename in natsorted(
glob.glob("C:\\Users\\Alex\\Desktop\\One Dimensional Reaction Diffusion Program\\data\\*.tiff")):
img = Image.open(filename)

# Cropping function
selected_region = (x1, y1, x2, y2) # selected region from previous user input
cropped_region = img.crop(selected_region) # taking opened image and cropping to selected region
if (y2 - y1) > (x2 - x1): # If cropped area is vertical, rotate into horizontal position
rotated_image = cropped_region.rotate(90, expand=1) # Expand 1 ensures image isn't cut off while rotating
else:
rotated_image = cropped_region # Does nothing if image is horizontal
img.close() # Closes the initially opened image

y_offset = 0 # Initial position is 0
new_im.paste(rotated_image, (0, y_offset)) # (x,y) indicates which direction to offset in
y_offset += rotated_image.size[1] # location of new image is y_offset (which had position 0) + the height of the image

new_im.save("C:\\Users\\Alex\\Desktop\\One Dimensional Reaction Diffusion Program\\output\\stacked.tiff", quality=95)

 

Highlighted
Commander Lvl 2
Message 8 of 9

Re: Memory Error on my crop and stack program

You're resetting y_offset to zero inside the loop. Initialize it to zero outside the loop and it'll be fine.

 

If you find it might help at all, add gc.collect() immediately after your img.close() line.

 

But it seems like you're good to go - congrats!

 

Please follow-up to let us know how you made out. For good karma, mark a reply as the answer if it helped!

Highlighted
Copilot Lvl 3
Message 9 of 9

Re: Memory Error on my crop and stack program

Thank you so much.